Hostname: page-component-5f56664f6-w4vgn Total loading time: 0 Render date: 2025-05-08T00:43:38.295Z Has data issue: false hasContentIssue false
  • English
  • Français

The Assessment of 21st Century Skills in Industrial and Organizational Psychology: Complex and Collaborative Problem Solving

Published online by Cambridge University Press:  26 March 2015

Jonas C. Neubert ,
Jakob Mainert ,
André Kretzschmar  and
Samuel Greiff
Jonas C. Neubert
Affiliation:
Institute of Cognitive Science and Assessment, University of Luxembourg, Luxembourg.
Jakob Mainert
Affiliation:
Institute of Cognitive Science and Assessment, University of Luxembourg, Luxembourg.
André Kretzschmar
Affiliation:
Institute of Cognitive Science and Assessment, University of Luxembourg, Luxembourg.
Samuel Greiff*
Affiliation:
Institute of Cognitive Science and Assessment, University of Luxembourg, Luxembourg.
*
Correspondence concerning this article should be addressed to Samuel Greiff, Institute of Cognitive Science and Assessment, University of Luxembourg, 6 Rue Richard Coudenhove Kalergi, 1359 Luxembourg-Kirchberg, Luxembourg. E-mail: [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

In this article, we highlight why and how industrial and organizational psychologists can take advantage of research on 21st century skills and their assessment. We present vital theoretical perspectives, a suitable framework for assessment, and exemplary instruments with a focus on advances in the assessment of human capital. Specifically, complex problem solving (CPS) and collaborative problem solving (ColPS) are two transversal skills (i.e., skills that span multiple domains) that are generally considered critical in the 21st century workplace. The assessment of these skills in education has linked fundamental research with practical applicability and has provided a useful template for workplace assessment. Both CPS and ColPS capture the interaction of individuals with problems that require the active acquisition and application of knowledge in individual or group settings. To ignite a discussion in industrial and organizational psychology, we discuss advances in the assessment of CPS and ColPS and propose ways to move beyond the current state of the art in assessing job-related skills.

Type
Focal Article
Information
Industrial and Organizational Psychology , Volume 8 , Issue 2 , June 2015 , pp. 238 - 268
Copyright
Copyright © Society for Industrial and Organizational Psychology 2015 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

References

Adkins, C. L., Russell, C. J., & Werbel, J. D. (1994). Judgments of fit in the selection process: The role of work value congruence. Personnel Psychology, 47, 605–623. doi:10.1111/j.1744-6570.1994.tb01740.xCrossRefGoogle Scholar
Aguinis, H., & Pierce, C. A. (2008). Enhancing the relevance of organizational behavior by embracing performance management research. Journal of Organizational Behavior, 29, 139–145. doi:10.1002/job.493Google Scholar
Allred, B. B., Snow, C. C., & Miles, R. E. (1996). Characteristics of managerial careers in the 21st century. Academy of Management Executive, 10 (4), 17–27. doi:10.5465/AME.1996.3145316Google Scholar
Anakwe, U. P., Hall, J. C., & Schor, S. M. (2000). Knowledge-related skills and effective career management. International Journal of Manpower, 21, 566–579. doi:10.1108/01437720010379024Google Scholar
Anderson, N. (2007). The practitioner-researcher divide revisited: Strategic-level bridges and the roles of IWO psychologists. Journal of Occupational and Organizational Psychology, 80, 175–183. doi:10.1348/096317907'187237CrossRefGoogle Scholar
Anderson, N., Herriot, P., & Hodgkinson, G. P. (2001). The practitioner-researcher divide in industrial, work and organizational (IWO) psychology: Where are we now, and where do we go from here? Journal of Occupational and Organizational Psychology, 74, 391–411. doi:10.1348/096317901167451CrossRefGoogle Scholar
Annett, J., & Duncan, K. (1967). Task analysis and training design. Occupational Psychology, 41, 211–221.Google Scholar
Armstrong, S. J., Cools, E., & Sadler-Smith, E. (2012). Role of cognitive styles in business and management: Reviewing 40 years of research: Role of cognitive styles in business and management. International Journal of Management Reviews, 14, 238–262. doi:10.1111/j.1468-2370.2011.00315.xCrossRefGoogle Scholar
Arthur, M. B., & Rousseau, D. M. (1996). A career lexicon for the 21st century. Academy of Management Perspectives, 10 (4), 28–39. doi:10.5465/AME.1996.3145317Google Scholar
Arun, N., Coyle, P. T., & Hauenstein, N. (2012). Learning agility: Still searching for clarity on a confounded construct. Industrial and Organizational Psychology: Perspectives on Science and Practice, 5, 290–293. doi:10.1111/j.1754-9434.2012.01447.xCrossRefGoogle Scholar
Autor, D. H., Katz, L. F., & Kearney, M. S. (2006). The polarization of the U.S. labor market. American Economic Review, 96 (2), 189–194. doi:10.1257/000282806777212620CrossRefGoogle Scholar
Autor, D. H., Levy, F., & Murnane, R. J. (2003). The skill content of recent technological change: An empirical exploration. The Quarterly Journal of Economics, 118, 1279–1333. doi:10.1162/003355303322552801CrossRefGoogle Scholar
Barrick, M. R., & Mount, M. K. (1991). The Big Five personality dimensions and job performance: A meta-analysis. Personnel Psychology, 44, 1–26. doi:10.1111/j.1744-6570.1991.tb00688.xCrossRefGoogle Scholar
Bass, B. M. (1974). The substance and the shadow. American Psychologist, 29, 870–886. doi:10.1037/h0037515CrossRefGoogle Scholar
Bass, B. M., & Avolio, B. J. (1990). Transformational leadership development: Manual for the multifactor leadership questionnaire. Palo Alto, CA: Consulting Psychologists Press.Google Scholar
Baumgarten, D., Geishecker, I., & Görg, H. (2010). Offshoring, tasks, and the skill-wage pattern. CEGE Discussion Paper, 98. Retrieved from http://ssrn.com/abstract=1574784. doi:10.2139/ssrn.1574784Google Scholar
Becker, S. O., Ekholm, K., & Muendler, M.-A. (2013). Offshoring and the onshore composition of tasks and skills. Journal of International Economics, 90, 91–106. doi:10.1016/j.jinteco.2012.10.005Google Scholar
Bird, A. (1994). Careers as repositories of knowledge: A new perspective on boundaryless careers. Journal of Organizational Behavior, 15, 325–344. doi:10.1002/job.4030150404CrossRefGoogle Scholar
Brannick, M. T., Levine, E. L., & Morgeson, F. P. (2007). Job and work analysis: Methods, research, and applications for human resource management (2nd ed.). Los Angeles, CA: Sage.Google Scholar
Brannick, M. T., & Prince, C. (1997). An overview of team performance measurement. In Brannick, M. T., Salas, E., & Prince, C. (Eds.), Team performance assessment and measurement: Theory, methods, and applications (pp. 3–16). Mahwah, NJ: Erlbaum.CrossRefGoogle Scholar
Braune, R., & Foshay, W. R. (1983). Towards a practical model of cognitive/information processing task analysis and schema acquisition for complex problem-solving situations. Instructional Science, 12, 121–145.Google Scholar
Brehmer, B. (1992). Dynamic decision making: Human control of complex systems. Acta Psychologica, 81, 211–241.Google Scholar
Brouwers, S. A., & Van De Vijver, F. J. R. (2012). Intelligence 2.0 in I–O psychology: Revival or contextualization? Industrial and Organizational Psychology: Perspectives on Science and Practice, 5, 158–160. doi:10.1111/j.1754-9434.2012.01422.xGoogle Scholar
Bruch, H., Spychala, A., & Wiegel, J. L. (2013). Wege zur hochleistungsorganisation: Wie die Graubündner Kantonalbank energie mobilisiert und erhält [Pathways to a high performance organization: How the Graubündner Kantonalbank mobilizes and maintains energy]. Personalführung, 2013 (2), 20–27.Google Scholar
Buchanan, D. A., & Huczynski, A. (2010). Organizational behaviour (7th ed.). Harlow, United Kingdom: Pearson Education.Google Scholar
Buchner, A. (1995). Basic topics and approaches to the study of complex problem solving. In Frensch, P. A. & Funke, J. (Eds.), Complex problem solving: The European perspective (pp. 27–63). Hillsdale, NJ: Erlbaum.Google Scholar
Caplan, R. D. (1987). Person–environment fit theory and organizations: Commensurate dimensions, time perspectives, and mechanisms. Journal of Vocational Behavior, 31, 248–267. doi:10.1016/0001-8791(87)90042-XGoogle Scholar
Cappelli, P. (2008). Talent management for the twenty-first century. Harvard Business Review, 86 (3), 74–81.Google Scholar
Cascio, W. F. (1995). Whither industrial and organizational psychology in a changing world of work? American Psychologist, 50, 928–939. doi:10.1037/0003-066X.50.11.928Google Scholar
Cascio, W. F. (1998). Managing human resources: Productivity, quality of work life, profits. Boston, MA: Irwin McGraw-Hill.Google Scholar
Cascio, W. F., & Aguinis, H. (2008). Research in industrial and organizational psychology from 1963 to 2007: Changes, choices, and trends. Journal of Applied Psychology, 93, 1062–1081. doi:10.1037/0021-9010.93.5.1062Google Scholar
Chan, D. (1996). Cognitive misfit of problem-solving style at work: A facet of person-organization fit. Organizational Behavior and Human Decision Processes, 68, 194–207. doi:10.1006/obhd.1996.0099Google Scholar
Chatman, J. A. (1989). Improving interactional organizational research: A Model of person-organization fit. Academy of Management Review, 14, 333–349. doi:10.5465/AMR.1989.4279063CrossRefGoogle Scholar
Clifford, J. P. (1994). Job analysis: Why do it, and how should it be done? Public Personnel Management, 23, 321–340. doi:10.1177/009102609402300211CrossRefGoogle Scholar
Cohen, D. J. (2007). The very separate worlds of academic and practitioner publications in human resource management: Reasons for the divide and concrete solutions for bridging the gap. Academy of Management Journal, 50, 1013–1019. doi:10.5465/AMJ.2007.27151946CrossRefGoogle Scholar
Cohen, M. D., March, J. G., & Olsen, J. P. (1972). “A garbage can model of organizational choice.” Administrative Science Quarterly, 17, 1–25.Google Scholar
Collings, D. G., & Mellahi, K. (2009). Strategic talent management: A review and research agenda. Human Resource Management Review, 19, 304–313. doi:10.1016/j.hrmr.2009.04.001CrossRefGoogle Scholar
Danner, D., Hagemann, D., Holt, D. V., Hager, M., Schankin, A., Wüstenberg, S., & Funke, J. (2011). Measuring performance in dynamic decision making. Journal of Individual Differences, 32, 225–233.CrossRefGoogle Scholar
Day, D. V., & Lord, R. G. (1988). Executive leadership and organizational performance: Suggestions for a new theory and methodology. Journal of Management, 14, 453–464. doi:10.1177/014920638801400308Google Scholar
De Meuse, K. P., Dai, G., & Hallenbeck, G. S. (2010). Learning agility: A construct whose time has come. Consulting Psychology Journal: Practice and Research, 62, 119–130. doi:10.1037/a0019988Google Scholar
DeRue, D. S., Ashford, S. J., & Myers, C. G. (2012). Learning agility: In search of conceptual clarity and theoretical grounding. Industrial and Organizational Psychology: Perspectives on Science and Practice, 5, 258–279. doi:10.1111/j.1754-9434.2012.01444.xGoogle Scholar
Dierdorff, E. C., & Wilson, M. A. (2003). A meta-analysis of job analysis reliability. Journal of Applied Psychology, 88, 635–646. doi:10.1037/0021-9010.88.4.635Google Scholar
Dries, N., Vantilborgh, T., & Pepermans, R. (2012). The role of learning agility and career variety in the identification and development of high potential employees. Personnel Review, 41, 340–358. doi:10.1108/00483481211212977CrossRefGoogle Scholar
Dunnette, M. D. (1990). Blending the science and practice of industrial and organizational psychology: Where are we and where are we going? In Dunnette, M. D. & Hough, L. M. (Eds.), Handbook of industrial and organizational psychology (2nd ed., Vol. 1, pp. 1–27). Palo Alto, CA: Consulting Psychologists Press.Google Scholar
Edwards, J. R. (1991). Person-job fit: A conceptual integration, literature review, and methodological critique. In Cooper, C. L. & Robertson, I. T. (Eds.), International Review of Industrial/Organizational Psychology (Vol. 9, pp. 383–357). London, United Kingdom: Wiley.Google Scholar
Eichinger, R. W., & Lombardo, M. M. (2004). Learning agility as a prime indicator of potential. Human Resource Planning, 27 (4), 12–15.Google Scholar
Fischer, A., Greiff, S., & Funke, J. (2012). The process of solving complex problems. Journal of Problem Solving, 4 (1), 19–42. doi:10.7771/1932-6246.1118Google Scholar
Fleishman, E. A. (1989). Manual for the Leadership Opinion Questionnaire (Rev. ed.). Rosemont, IL: National Computer Systems.Google Scholar
Fleishman, E. A., & Reilly, M. E. (1992). Handbook of human abilities: Definitions, measurements, and job task requirements. Palo Alto, CA: Consulting Psychologists Press.Google Scholar
Funke, J. (2001). Dynamic systems as tools for analysing human judgement. Thinking & Reasoning, 7, 69–89. doi:10.1080/13546780042000046Google Scholar
Funke, J. (2010). Complex problem solving: A case for complex cognition? Cognitive Processing, 11, 133–142. doi:10.1007/s10339-009-0345-0Google Scholar
Gonzalez, C., Lerch, J. F., & Lebiere, C. (2003). Instance-based learning in dynamic decision making. Cognitive Science, 27, 591–635.Google Scholar
Gonzalez, C., Vanyukov, P., & Martin, M. K. (2005). The use of microworlds to study dynamic decision making. Computers in Human Behavior, 21, 273–286.Google Scholar
Goos, M., & Manning, A. (2007). Lousy and lovely jobs: The rising polarization of work in Britain. Review of Economics and Statistics, 89, 118–133. doi:10.1162/rest.89.1.118CrossRefGoogle Scholar
Goos, M., Manning, A., & Salomons, A. (2009). Job polarization in Europe. The American Economic Review, 99 (2), 58–63.Google Scholar
Gottfredson, L. S. (1997). Why g matters: The complexity of everyday life. Intelligence, 24, 79–132. doi:10.1016/S0160-2896(97)90014-3CrossRefGoogle Scholar
Greenhaus, J. H., Callanan, G. A., & Kaplan, E. (1995). The role of goal setting in career management. International Journal of Career Management, 7 (5), 3–12. doi:10.1108/09556219510093285CrossRefGoogle Scholar
Greiff, S. (2012). From interactive to collaborative problem solving: Current issues in the Programme for International Student Assessment. Review of Psychology, 19, 111–121.Google Scholar
Greiff, S., Fischer, A., Wüstenberg, S., Sonnleitner, P., Brunner, M., & Martin, R. (2013). A multitrait–multimethod study of assessment instruments for complex problem solving. Intelligence, 41, 579–596. doi:10.1016/j.intell.2013.07.012Google Scholar
Greiff, S., Kretzschmar, A., Müller, J. C., Spinath, B., & Martin, R. (2014). Computer-based assessment of complex problem solving in educational contexts and how it is influenced by students’ level of information and communication technology literacy. Journal of Educational Psychology, 106, 666–680. doi:10.1037/a0035426CrossRefGoogle Scholar
Greiff, S., & Neubert, J. C. (2014). On the relation of complex problem solving, personality, fluid intelligence, and academic achievement. Learning and Individual Differences, 36, 37–48. doi:10.1016/j.lindif.2014.08.003CrossRefGoogle Scholar
Greiff, S., & Wüstenberg, S. (2014). Assessment with microworlds using MicroDYN: Measurement invariance and latent mean comparisons: Psychometric properties across several student samples and blue-collar workers. European Journal of Psychological Assessment, 30, 304–314. doi:10.1027/1015-5759/a000194Google Scholar
Greiff, S., Wüstenberg, S., & Funke, J. (2012). Dynamic problem solving: A new assessment perspective. Applied Psychological Measurement, 36, 189–213. doi:10.1177/0146621612439620Google Scholar
Greiff, S., Wüstenberg, S., Molnár, G., Fischer, A., Funke, J., & Csapó, B. (2013). Complex problem solving in educational contexts—Something beyond g: Concept, assessment, measurement invariance, and construct validity. Journal of Educational Psychology, 105, 364–379. doi:10.1037/a0031856CrossRefGoogle Scholar
Griffin, P., McGaw, B., & Care, E. (Eds.). (2012). Assessment and teaching of 21st century skills. New York, NY: Springer.CrossRefGoogle Scholar
Grossman, G. M., & Rossi-Hansberg, E. (2008). Trading tasks: A simple theory of offshoring. American Economic Review, 98 (5), 1978–1997. doi:10.1257/aer.98.5.1978Google Scholar
Hall, D. T., & Mirvis, P. H. (1996). The new protean career. In Hall, D. T. (Ed.), The career is dead—Long live the career: A relational approach to careers (pp. 15–45). San Francisco, CA: Jossey-Bass.Google Scholar
Harvey, D. F., & Bowin, R. B. (1996). Human resource management: An experiential approach. Englewood Cliffs, NJ: Prentice-Hall.Google Scholar
Hogan, J., & Holland, B. (2003). Using theory to evaluate personality and job-performance relations: A socioanalytic perspective. Journal of Applied Psychology, 88, 100–112. doi:10.1037/0021-9010.88.1.100Google Scholar
Holland, J. L. (1985). Making vocational choices: A theory of vocational personalities and work environments. Odessa, FL: Psychological Assessment Resources.Google Scholar
Hunter, J. E., & Schmidt, F. L. (1996). Intelligence and job performance: Economic and social implications. Psychology, Public Policy, and Law, 2, 447–472. doi:10.1037/1076-8971.2.3-4.447Google Scholar
Jacobs, T. O., & Jaques, E. (1987). Leadership in complex systems. In Zeidner, J. (Ed.), Human productivity enhancement (Vols. 1–2, pp. 7–65). New York, NY: Praeger.Google Scholar
Jones, G., & Schneider, W. J. (2006). Intelligence, human capital, and economic growth: A Bayesian averaging of classical estimates (BACE) approach. Journal of Economic Growth, 11, 71–93. doi:10.1007/s10887-006-7407-2CrossRefGoogle Scholar
Keane, A. J., & Nair, P. B. (2001). Problem solving environments in aerospace design. Advances in Engineering Software, 32, 477–487. doi:10.1016/S0965-9978(00)00108-3Google Scholar
Kirton, M. (1976). Adaptors and innovators: A description and measure. Journal of Applied Psychology, 61, 622–629. doi:10.1037/0021-9010.61.5.622Google Scholar
Klein, C., DeRouin, R. E., & Salas, E. (2006). Uncovering workplace interpersonal skills: A review, framework, and research agenda. In Hodgkinson, G. P. & Ford, J. K. (Eds.), International review of industrial and organizational psychology (pp. 79–126). West Sussex, England: Wiley.Google Scholar
Klein, G. (2008). Naturalistic decision making. Human Factors: The Journal of the Human Factors and Ergonomics Society, 50, 456–460. doi:10.1518/001872008´288385CrossRefGoogle ScholarPubMed
Klein, G., Orasanu, J., Calderwood, R., & Zsambok, C. E. (1993). Decision making in action: Models and methods. Norwood, NJ: Ablex.Google Scholar
Kristof, A. L. (1996). Person–organization fit: An integrative review of its conceptualizations, measurement, and implications. Personnel Psychology, 49, 1–49. doi:10.1111/j.1744-6570.1996.tb01790.xGoogle Scholar
Levine, E. L., Ash, R. A., Hall, H., & Sistrunk, F. (1983). Evaluation of job analysis methods by experienced job analysts. The Academy of Management Journal, 26, 339–348. doi:10.2307/255981Google Scholar
Levine, E. L., Sistrunk, F., McNutt, K. J., & Gael, S. (1988). Exemplary job analysis systems in selected organizations: A description of process and outcomes. Journal of Business and Psychology, 3, 3–21. doi:10.1007/BF01016745CrossRefGoogle Scholar
Lewis, R. E., & Heckman, R. J. (2006). Talent management: A critical review. Human Resource Management Review, 16, 139–154. doi:10.1016/j.hrmr.2006.03.001Google Scholar
Lombardo, M. M., & Eichinger, R. W. (2000). High potentials as high learners. Human Resource Management, 39, 321–329. doi:10.1002/1099-050X(200024)39:4<321::AID-HRM4>3.0.CO;2-1Google Scholar
Marshall-Mies, J. C., Fleishman, E. A., Martin, J. A., Zaccaro, S. J., Baughman, W. A., & McGee, M. L. (2000). Development and evaluation of cognitive and metacognitive measures for predicting leadership potential. The Leadership Quarterly, 11, 135–153. doi:10.1016/S1048-9843(99)00046-6CrossRefGoogle Scholar
McCormick, E. J., Jeanneret, P. R., & Mecham, R. C. (1972). A study of job characteristics and job dimensions as based on the Position Analysis Questionnaire (PAQ). Journal of Applied Psychology, 56, 347–368. doi:10.1037/h0033099CrossRefGoogle Scholar
McEntire, L. E., Dailey, L. R., Osburn, H. K., & Mumford, M. D. (2006). Innovations in job analysis: Development and application of metrics to analyze job data. Human Resource Management Review, 16, 310–323. doi:10.1016/j.hrmr.2006.05.004CrossRefGoogle Scholar
Middleton, H. (2002). Complex problem solving in a workplace setting. International Journal of Educational Research, 37, 67–84. doi:10.1016/S0883-0355(02)00022-8Google Scholar
Mumford, M. D., Mobley, M. I., Reiter-Palmon, R., Uhlman, C. E., & Doares, L. M. (1991). Process analytic models of creative capacities. Creativity Research Journal, 4, 91–122. doi:10.1080/10400419109534380Google Scholar
Mumford, M. D., Zaccaro, S. J., Harding, F. D., Jacobs, T. O., & Fleishman, E. A. (2000). Leadership skills for a changing world: Solving complex social problems. The Leadership Quarterly, 11, 11–35. doi:10.1016/S1048-9843(99)00041-7Google Scholar
Murphy, K. R., & Saal, F. E. (1990). What should we expect from scientist–practitioners? In Murphy, K. R. & Saal, F. E. (Eds.), Psychology in organizations: Integrating science and practice (pp. 49–66). Hillsdale, NJ: Erlbaum.Google Scholar
Naikar, N., Moylan, A., & Pearce, B. (2006). Analysing activity in complex systems with cognitive work analysis: Concepts, guidelines and case study for control task analysis. Theoretical Issues in Ergonomics Science, 7, 371–394. doi:10.1080/14639220500098821CrossRefGoogle Scholar
National Center for O*NET Development. (2009). New and emerging occupations of the 21st century: Updating the O*NET-SOC taxonomy [O*NET Reports and Documents]. Raleigh, NC: Author.Google Scholar
National Research Council. (2012). Education for life and work: Developing transferable knowledge and skills in the 21st century. Washington, DC: The National Academies Press.Google Scholar
Novick, L. R., & Bassok, M. (2005). Problem solving. In Holyoak, K. J. & Morrison, R. G. (Eds.), The Cambridge handbook of thinking and reasoning (pp. 321–349). New York, NY: Cambridge University Press.Google Scholar
O’Neil, H. F., Chuang, S., & Chung, G. K. W. K. (2004). Issues in the computer-based assessment of collaborative problem solving (CSE Report No. 620). Los Angeles, CA: National Center for Research on Evaluation, Standards, and Student Testing.Google Scholar
O’Reilly, C. A., Chatman, J. A., & Caldwell, D. F. (1991). People and organizational culture: A profile comparison approach to assessing person–organization fit. Academy of Management Journal, 34, 487–516. doi:10.2307/256404Google Scholar
Organisation for Economic Co-operation and Development. (2013a). Employment by size class. In Entrepreneurship at a glance 2013. Paris, France: Author. Retrieved from http://www.keepeek.com/Digital-Asset-Management/oecd/industry-and-services/entrepreneurship-at-a-glance-2013/employment-by-size-class_entrepreneur_aag-2013-8-en#page1Google Scholar
Organisation for Economic Co-operation and Development. (2013b). OECD skills outlook 2013: First results from the survey of adult skills. Paris, France: Author. doi:10.1787/9789264204256-enGoogle Scholar
Organisation for Economic Co-operation and Development. (2013c). PISA 2012 assessment and analytical framework. Paris, France: Author.Google Scholar
Organisation for Economic Co-operation and Development. (2013d). PISA 2015 draft collaborative problem solving framework. Paris, France: Author. Retrieved from http://www.oecd.org/pisa/pisaproducts/Draft%20PISA%202015%20Collaborative%20Problem%20Solving%20Framework%20.pdfGoogle Scholar
Osman, M. (2010). Controlling uncertainty: A review of human behavior in complex dynamic environments. Psychological Bulletin, 136, 65–86. doi:10.1037/a0017815Google Scholar
Oswald, F. L., & Hough, L. (2012). I–O 2.0 From intelligence 1.5: Staying (just) behind the cutting edge of intelligence theories. Industrial and Organizational Psychology, 5, 172–175. doi:10.1111/j.1754-9434.2012.01425.xGoogle Scholar
Pearlman, K. (1980). Job families: A review and discussion of their implications for personnel selection. Psychological Bulletin, 87, 1–28. doi:10.1037/0033-2909.87.1.1Google Scholar
Putz-Osterloh, W. (1981). Über die beziehung zwischen Testintelligenz und problemlöseerfolg [The relation between test intelligence and problem solving success]. Zeitschrift Für Psychologie Mit Zeitschrift Für Angewandte Psychologie, 189 (1), 79–100.Google Scholar
Rasmussen, J., Pejtersen, A. M., & Goodstein, L. P. (1994). Cognitive systems engineering. New York, NY: Wiley.Google Scholar
Ready, D. A., & Conger, J. A. (2007). Make your company a talent factory. Harvard Business Review, 85 (6), 68–77.Google Scholar
Reiter-Palmon, R., & Illies, J. J. (2004). Leadership and creativity: Understanding leadership from a creative problem-solving perspective. The Leadership Quarterly, 15, 55–77. doi:10.1016/j.leaqua.2003.12.005Google Scholar
Riding, R. J. (1997). On the nature of cognitive style. Educational Psychology, 17, 29–49. doi:10.1080/0144341970170102Google Scholar
Rupp, D. E., & Beal, D. (2007, July). Checking in with the scientist–practitioner model: How are we doing? The Industrial-Organizational Psychologist, 45 (1), 35–40.Google Scholar
Rynes, S. L., Colbert, A. E., & Brown, K. G. (2002). HR professionals’ beliefs about effective human resource practices: Correspondence between research and practice. Human Resource Management, 41, 149–174. doi:10.1002/hrm.10029Google Scholar
Rynes, S. L., & Gerhart, B. (1990). Interviewer assessments of applicant “fit”: An exploratory investigation. Personnel Psychology, 43, 13–35. doi:10.1111/j.1744-6570.1990.tb02004.xGoogle Scholar
Rynes, S. L., Giluk, T. L., & Brown, K. G. (2007). The very separate worlds of academic and practitioner periodicals in human resource management: Implications for evidence-based management. Academy of Management Journal, 50, 987–1008. doi:10.5465/AMJ.2007.27151939Google Scholar
Scherbaum, C. A., Goldstein, H. W., Yusko, K. P., Ryan, R., & Hanges, P. J. (2012). Intelligence 2.0: Reestablishing a research program on g in I–O psychology. Industrial and Organizational Psychology, 5, 128–148. doi:10.1111/j.1754-9434.2012.01419.xCrossRefGoogle Scholar
Schippmann, J. S., Ash, R. A., Battista, M., Carr, L., Eyde, L. D., Hesketh, B., . . . Sanchez, J. I. (2000). The Practice of Competency Modeling. Personnel Psychology, 53, 703–740.CrossRefGoogle Scholar
Schmid, U., Ragni, M., Gonzalez, C., & Funke, J. (2011). The challenge of complexity for cognitive systems. Cognitive Systems Research, 12, 211–218. doi:10.1016/j.cogsys.2010.12.007Google Scholar
Schmidt, F. L., & Hunter, J. E. (1998). The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. Psychological Bulletin, 124, 262–274. doi:10.1037/0033-2909.124.2.262CrossRefGoogle Scholar
Schmidt, F. L., Hunter, J. E., Outerbridge, A. N., & Goff, S. (1988). Joint relation of experience and ability with job performance: Test of three hypotheses. Journal of Applied Psychology, 73, 46–57. doi:10.1037/0021-9010.73.1.46Google Scholar
Schneider, B., & Konz, A. M. (1989). Strategic job analysis. Human Resource Management, 28, 51–63. doi:10.1002/hrm.3930280104Google Scholar
Schooler, C., Mulatu, M. S., & Oates, G. (1999). The continuing effects of substantively complex work on the intellectual functioning of older workers. Psychology and Aging, 14, 483–506. doi:10.1037/0882-7974.14.3.483Google Scholar
Schweizer, F., Wüstenberg, S., & Greiff, S. (2013). Validity of the MicroDYN approach: Complex problem solving predicts school grades beyond working memory capacity. Learning and Individual Differences, 24, 42–52. doi:10.1016/j.lindif.2012.12.011Google Scholar
Seibert, S. E., & Kraimer, M. L. (2001). The five-factor model of personality and career success. Journal of Vocational Behavior, 58, 1–21. doi:10.1006/jvbe.2000.1757CrossRefGoogle Scholar
Shepherd, A. (2001). Hierarchical task analysis. New York, NY: Taylor & Francis.Google Scholar
Siddique, C. M. (2004). Job analysis: A strategic human resource management practice. International Journal of Human Resource Management, 15, 219–244. doi:10.1080/0958519032000157438CrossRefGoogle Scholar
Smith, G. F. (1997). Managerial problem solving: A problem-centered approach. In Zsambok, C. E. & Klein, G. (Eds.), Naturalistic decision making (pp. 371–380). Hillsdale, NJ: Erlbaum.Google Scholar
Sonnleitner, P., Brunner, M., Greiff, S., Funke, J., Keller, U., Martin, R., . . . Latour, T. (2012). The genetics lab: Acceptance and psychometric characteristics of a computer-based microworld assessing complex problem solving. Psychological Test and Assessment Modeling, 54, 54–72.Google Scholar
Sonnleitner, P., Keller, U., Martin, R., & Brunner, M. (2013). Students’ complex problem-solving abilities: Their structure and relations to reasoning ability and educational success. Intelligence, 41, 289–305. doi:10.1016/j.intell.2013.05.002Google Scholar
Spitz-Oener, A. (2006). Technical change, job tasks, and rising educational demands: Looking outside the wage structure. Journal of Labor Economics, 24, 235–270.Google Scholar
Taylor, F. W. (1911). The principles of scientific management. New York, NY: Harper.Google Scholar
Taylor, W. G. K. (1989). The Kirton Adaption–Innovation Inventory: A re-examination of the factor structure. Journal of Organizational Behavior, 10, 297–307. doi:10.1002/job.4030100402CrossRefGoogle Scholar
Tushman, M., & O’Reilly, C. (2007). Research and relevance: Implications of Pasteur's quadrant for doctoral programs and faculty development. Academy of Management Journal, 50, 769–774. doi:10.5465/AMJ.2007.26279169Google Scholar
Vancouver, J. B., & Schmitt, N. W. (2006). An exploratory examination of person-organization fit: Organizational goal congruence. Personnel Psychology, 44, 333–352. doi:10.1111/j.1744-6570.1991.tb00962.xGoogle Scholar
Vargas Cortes, M. G., & Beruvides, M. G. (1996). An analysis of middle management work in non-steady conditions. Computers & Industrial Engineering, 31, 53–57. doi:10.1016/0360-8352(96)00077-0Google Scholar
Vicente, K. J. (1999). Cognitive work analysis: Toward safe, productive, and healthy computer-based work. Hoboken, NJ: Taylor & Francis.Google Scholar
Warr, P., & Bunce, D. (1995). Trainee characteristics and the outcomes of open learning. Personnel Psychology, 48, 347–375. doi:10.1111/j.1744-6570.1995.tb01761.xGoogle Scholar
Weede, E., & Kämpf, S. (2002). The impact of intelligence and institutional improvements on economic growth. Kyklos, 55, 361–380. doi:10.1111/1467-6435.00191CrossRefGoogle Scholar
Wüstenberg, S., Greiff, S., & Funke, J. (2012). Complex problem solving—More than reasoning? Intelligence, 40, 1–14.Google Scholar
Zaccaro, S. J., Mumford, M. D., Connelly, M. S., Marks, M. A., & Gilbert, J. A. (2000). Assessment of leader problem-solving capabilities. The Leadership Quarterly, 11, 37–64. doi:10.1016/S1048-9843(99)00042-9Google Scholar
Zaccaro, S. J., Mumford, M. D., Marks, M. A., Connelly, M. S., Threlfall, K. V., Gilbert, J. A., & Fleishman, E. A. (1997). Cognitive and temperament determinants of Army leadership (U.S. Army Research Institute for the Behavioral and Social Sciences Tech. Rep. No. MRI 95-4). Bethesda, MD: Management Research Institute.Google Scholar
Zsambok, C. E., & Klein, G. (Eds.). (1997). Naturalistic decision making. Hillsdale, NJ: Erlbaum.Google Scholar