Hostname: page-component-586b7cd67f-t8hqh Total loading time: 0 Render date: 2024-11-22T18:00:10.606Z Has data issue: false hasContentIssue false
  • English
  • Français

Feature Engineering for Design Thinking Assessment

Part of: User-Centred Design

Published online by Cambridge University Press:  26 July 2019

Ryan Arlitt ,
Sumbul Khan  and
Lucienne Blessing
Ryan Arlitt*
Affiliation:
Technical University of Denmark;
Sumbul Khan
Affiliation:
Singapore University of Technology and Design
Lucienne Blessing
Affiliation:
Singapore University of Technology and Design
*
Contact: Arlitt, Ryan Michael, Technical University of Denmark, Mechanical Engineering, Denmark, [email protected]

Abstract

Core share and HTML view are not available for this content. However, as you have access to this content, a full PDF is available via the ‘Save PDF’ action button.

As design and design thinking become increasingly important competencies for a modern workforce, the burden of assessing these fuzzy skills creates a scalability bottleneck. Toward addressing this need, this paper presents an exploratory study into a scalable computational approach for design thinking assessment. In this study, student responses to a variety of contextualized design questions – gathered both before and after participation in a design thinking training course – are analyzed. Specifically, a variety of text features are engineered, tested, and interpreted within a design thinking framework in order to identify specific markers of design thinking skill acquisition. Key findings of this work include identification of text features that may enable scalable measurement of (1) user-centric language and (2) design thinking concept acquisition. These results contribute toward the creation of computational tools to ease the burden of providing feedback about design thinking skills to a wide audience.

Keywords

Design thinkingDesign educationMachine learningSemantic data processing
Type
Article
Information
Proceedings of the Design Society: International Conference on Engineering Design , Volume 1 , Issue 1 , July 2019 , pp. 3891 - 3900
Creative Commons
Creative Common License - CCCreative Common License - BYCreative Common License - NCCreative Common License - ND
This is an Open Access article, distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivatives licence (http://creativecommons.org/licenses/by-nc-nd/4.0/), which permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is unaltered and is properly cited. The written permission of Cambridge University Press must be obtained for commercial re-use or in order to create a derivative work.
Copyright
© The Author(s) 2019

References

Abdi, H. and Williams, L.J. (2010), “Principal Component Analysis”, WIREs Comput. Stat., Vol. 2 No. 4, pp. 433–459, John Wiley & Sons, Inc., New York, NY, USA.Google Scholar
Aflatoony, L., Wakkary, R. and Neustaedter, C. (2018), “Becoming a Design Thinker: Assessing the Learning Process of Students in a Secondary Level Design Thinking Course”, International Journal of Art & Design Education, Vol. 37 No. 3, pp. 438–453, Wiley/Blackwell (10.1111).Google Scholar
Balfour, S.P. (2013), “Assessing Writing in MOOCs: Automated Essay Scoring and Calibrated Peer ReviewTM.”, Research & Practice in Assessment, ERIC, Vol. 8, pp. 40–48.Google Scholar
Bartholomew, S.R. (2017), “Assessing open-ended design problems”, Technology and Engineering Teacher, Vol. 76 No. 6, pp. 13–17.Google Scholar
Beyerlein, S., Davis, D., Trevisan, M., Harrison, K. and Thompson, L. (2006), “Assessment Framework for Capstone Design Courses”, Proceedings of American Society for Engineering Education Annual Conference.Google Scholar
Blei, D.M., Ng, A.Y. and Jordan, M.I. (2003), “Latent dirichlet allocation”, Journal of Machine Learning Research : JMLR, Vol. 3, pp. 993–1022.Google Scholar
Blizzard, J., Klotz, L., Potvin, G., Hazari, Z., Cribbs, J. and Godwin, A. (2015), “Using survey questions to identify and learn more about those who exhibit design thinking traits”, Design Studies, Vol. 38, pp. 92–110. Elsevier Ltd.Google Scholar
Camburn, B.A., Auernhammer, J.M., Sng, K.H.E., Mignone, P.J., Arlitt, R.M., Perez, K.B., Huang, Z., et al. (2017), “Design Innovation: A Study of Integrated Practice”, Volume 7: 29th International Conference on Design Theory and Methodology, pp. 1–10.Google Scholar
Carroll, M., Goldman, S., Britos, L., Koh, J., Royalty, A. and Hornstein, M. (2010), “Destination, imagination and the fires within: Design thinking in a middle school classroom”, International Journal of Art and Design Education, Available at: https://doi.org/10.1111/j.1476-8070.2010.01632.xGoogle Scholar
Christensen, K.S., Hjorth, M., Iversen, O.S. and Blikstein, P. (2016), “Towards a formal assessment of design literacy: Analyzing K-12 students’ stance towards inquiry”, Design Studies, Vol. 46, pp. 125–151.Google Scholar
Cross, N. (2011), Design Thinking : Understanding How Designers Think and Work, Berg.Google Scholar
Davis, D.C., Gentili, K.L., Trevisan, M.S. and Calkins, D.E. (2002), “Engineering design assessment processes and scoring scales for program improvement and accountability”, Journal of Engineering Education, Vol. 91 No. 2, pp. 211–221, Blackwell Publishing Ltd.Google Scholar
Deerwester, S., Dumais, S.T., Furnas, G.W., Landauer, T.K. and Harshman, R. (1990), “Indexing by latent semantic analysis”, Journal of the American Society for Information Science, Vol. 41 No. 6, pp. 391–407, Wiley Online Library.Google Scholar
Demirkan, H. and Afacan, Y. (2012), “Assessing creativity in design education: Analysis of creativity factors in the first-year design studio”, Design Studies, Vol. 33 No. 3, pp. 262–278, Elsevier Ltd.Google Scholar
DesignSingapore Council (2016), Design 2025.Google Scholar
Diefes-Dux, H.A., Zawojewski, J.S. and Hjalmarson, M.A. (2010), “Using Educational Research in the Design of Evaluation Tools for Open-Ended Problems”, International Journal of Engineering Education, Vol. 26 No. 4, pp. 807–819, SI.Google Scholar
Domingos, P. (2012), “A few useful things to know about machine learning”, Communications of the ACM, ACM, Vol. 55 No. 10, pp. 78–87.Google Scholar
Doppelt, Y. (2009), “Assessing creative thinking in design-based learning”, International Journal of Technology and Design Education, Vol. 19 No. 1, pp. 55–65.Google Scholar
Jonassen, D., Strobel, J. and Lee, C.B. (2006), “Everyday problem solving in engineering: Lessons for engineering educators”, Journal of Engineering Education, Available at: https://doi.org/10.1002/j.2168-9830.2006.tb00885.xGoogle Scholar
Kelley, T. and Kelley, D. (2013), “Creative confidence: Unleashing the creative potential within us all”, The Business Source, Available at: https://doi.org/10.4103/0255-0857.93014Google Scholar
Kimbell, R. (2012), “Evolving project e-scape for national assessment”, International Journal of Technology and Design Education, Vol. 22 No. 2, pp. 135–155, Springer Netherlands.Google Scholar
Kluyver, T., Ragan-Kelley, B., Pérez, F., Granger, B.E., Bussonnier, M., Frederic, J., Kelley, K., et al. (2016), “Jupyter Notebooks-a publishing format for reproducible computational workflows”, ELPUB, pp. 87–90.Google Scholar
Le, Q.V. and Mikolov, T. (2014), “Distributed Representations of Sentences and Documents”, CoRR, abs/1405.4, Available at: http://arxiv.org/abs/1405.4053.Google Scholar
Mc Laughlin, G.H. (1969), “SMOG grading-a new readability formula”, Journal of Reading, Vol. 12 No. 8, pp. 639–646, JSTOR.Google Scholar
McLaren, S.V. and Stables, K. (2008), “Exploring key discriminators of progression: relationships between attitude, meta-cognition and performance of novice designers at a time of transition”, Design Studies, Vol. 29 No. 2, pp. 181–201.Google Scholar
Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S. and Dean, J. (2013), “Distributed representations of words and phrases and their compositionality”, Advances in Neural Information Processing Systems, pp. 3111–3119.Google Scholar
Miller, T. (2003), “Essay assessment with latent semantic analysis”, Journal of Educational Computing Research, Vol. 29 No. 4, pp. 495–512, SAGE Publications Sage CA, Los Angeles, CA.Google Scholar
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., et al. (2011), “Scikit-learn: Machine Learning in {P}ython”, Journal of Machine Learning Research, Vol. 12, pp. 2825–2830.Google Scholar
Řehůřek, R. and Sojka, P. (2010), “Software Framework for Topic Modelling with Large Corpora”, Proceedings of the LREC 2010 Workshop on New Challenges for NLP Frameworks, ELRA, Valletta, Malta, pp. 45–50.Google Scholar
Shermis, M.D. and Burstein, J. (2013), Handbook of Automated Essay Evaluation: Current Applications and New Directions, Routledge.Google Scholar
Sparck Jones, K. (1972), “A statistical interpretation of term specificity and its application in retrieval”, Journal of Documentation, MCB UP Ltd, Vol. 28 No. 1, pp. 11–21.Google Scholar
SUTD-MIT International Design Centre (2018), Design Odyssey, Available at: https://idc.sutd.edu.sg/programmes/design-odyssey/.Google Scholar
“Textacy” (2018), Chartbeat Labs, Available at: https://github.com/chartbeat-labs/textacy.Google Scholar
Wrigley, C., Mosely, G. and Tomitsch, M. (2018), “Design Thinking Education: A Comparison of Massive Open Online Courses”, She Ji: The Journal of Design, Economics, and Innovation, Elsevier, Vol. 4 No. 3, pp. 275–292.Google Scholar
Wrigley, C. and Straker, K. (2017), “Design Thinking pedagogy: the Educational Design Ladder”, Innovations in Education and Teaching International, Available at: https://doi.org/10.1080/14703297.2015.1108214Google Scholar