Hostname: page-component-cd9895bd7-8ctnn Total loading time: 0 Render date: 2024-12-24T16:40:32.260Z Has data issue: false hasContentIssue false
  • English
  • Français

Design rationale: Researching under uncertainty

Published online by Cambridge University Press:  18 September 2008

Janet E. Burge
Janet E. Burge
Affiliation:
Computer Science and Systems Analysis Department, Miami University, Oxford, Ohio, USA
Get access Rights & Permissions [Opens in a new window]

Abstract

Rationale research in software development is a challenging area because although there is no shortage of advocates for its value, there is also no shortage of reasons for why rationale is unlikely to be captured in practice. Despite more than 30 years of research there still remains much uncertainty: how useful are the potential benefits and how insurmountable are the barriers? Will the value of the rationale (design and otherwise) justify the cost of collecting it? Although there have been numerous rationale research projects, many, if not most, received little or no empirical evaluation. There also have not been many studies examining what the needs are of the practitioners who would be supported by the rationale. This article discusses the “doom and gloom” predictions of rationale's failure, provides a survey of evaluations of rationale systems, and discusses what we hope is a brighter outlook for rationale research in the future. There are development standards and synergistic research areas that may help with rationale research and its acceptance in the software community with which we should be working. This article also presents the results of a pilot survey of software developers who were asked how they would envision using rationale and what they believe the most important barriers are. Although some results were as expected, there were also some surprises. Research on technology transfer indicates that, among other things, to transition successfully from research into practice we need to understand the need that is being met and demonstrate the value of our approach. Until we have determined how our work is needed by the people we are trying to help we will remain researching under uncertainty.

Keywords

Design RationaleEmpirical StudiesSoftware EngineeringTraceability
Type
Research Article
Information
AI EDAM , Volume 22 , Issue 4: Design Rationale , November 2008 , pp. 311 - 324
Copyright
Copyright © Cambridge University Press 2008

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

REFERENCES

Bahler, D., & Bowen, J. (1992). Design rationale management in concurrent engineering, Proc. Workshop on Design Rationale Capture and Use, 10th National Conf. Artificial Intelligence.Google Scholar
Ball, L., Lambell, N., Ormerod, T.C., Slavin, S., & Mariani, J. (1999). Representing design rationale to support innovative design reuse: a minimalist approach. Proc. 4th Design Thinking Research Symp., pp. I.75–I.87.Google Scholar
Baniassad, E.L., Murphy, G.C., & Schwanninger, C. (2003). Design pattern rationale graphs: linking design to source. Proc. 25th ICSE, pp. 352–362.CrossRefGoogle Scholar
Barbar, M., Gorton, I., & Kitchenham, B. (2006). A framework for supporting architecture knowledge and rationale management. In Rationale Management in Software Engineering (Dutoit, A., McCall, R., Mistrík, I., & Paech, B., Eds.), pp. 237–254. Heidelberg: Springer–Verlag.CrossRefGoogle Scholar
Barber, K.S., & Graser, T. (2000). Reference Architecture Representation Environment (RARE), A Tool to Support Object-Oriented Software Architecture Derivation and Evaluation, Technical Report TR00-UT-LIPS-SEPA-04. Austin, TX: University of Texas at Austin, Laboratory for Intelligent Processes and Systems.Google Scholar
Bass, L., Clements, P., Nord, R.L., & Stafford, J. (2006). Capturing and using rationale for a software architecture. In Rationale Management in Software Engineering (Dutoit, A., McCall, R., Mistrík, I., & and Paech, B., Eds.), pp. 255–272. Heidelberg: Springer–Verlag.CrossRefGoogle Scholar
Boehm, B. (2006). Value-based software engineering: overview and agenda. In Value-Based Software Engineering (Biffl, S., Aurum, A., Boehm, B., Erdogmus, H., & Grünbacher, P., Eds.), pp. 3–14. Heidelberg: Springer–Verlag.CrossRefGoogle Scholar
Boehm, B., & Jain, A. (2006). An initial theory of value-based software engineering. In Value-Based Software Engineering (Biffl, S., Aurum, A., Boehm, B., Erdogmus, H., & Grünbacher, P., Eds.), pp. 15–37. Heidelberg: Springer–Verlag.CrossRefGoogle Scholar
Boehm, B., & Kitapci, H. (2006). The WinWin approach: using a requirements negotiation tool for rationale capture and use. In Rationale Management in Software Engineering (Dutoit, A., McCall, R., Mistrík, I., & Paech, B., Eds.), pp. 173–190. Heidelberg: Springer–Verlag.CrossRefGoogle Scholar
Boehm, B., & Ross, R. (1989). Theory-W software project management: principles and examples. IEEE Transactions on Software Engineering 18(7), 902–916.CrossRefGoogle Scholar
Bracewell, R., Ahmed, S., & Wallace, K. (2004). DREd and Design Folders, a way of capturing, storing, and passing on knowledge generated during design projects. Proc. ASME 2004 Design Automation Conf.CrossRefGoogle Scholar
Brandish, M.J., Hague, M.J., & Taleb-Bendiab, A. (1996). M-LAP: a machine learning apprentice agent for computer supported design. AID’96 Machine Learning in Design Workshop.Google Scholar
Bratthall, L., Johansson, E., & Regnel, B. (2000). Is a design rationale vital when predicting change impact?—a controlled experiment on software architecture evolution. Proc. Int. Conf. Product Focused Software Process Improvement, pp. 126–139.CrossRefGoogle Scholar
Brice, A., & Johns, B. (1999). Improving Design by Improving the Design Process, Technical Report QSL-9002A-WP-001.Google Scholar
Brissaud, D., Garro, O., & Poveda, O. (2003). Design process rationale capture and support by abstraction of criteria. Research in Engineering Design 14, 162–172.CrossRefGoogle Scholar
Britt, B., & Glagowski, T. (1996). Reconstructive derivational analogy: a machine learning approach to automating redesign. Artificial Intelligence for Engineering Design, Analysis and Manufacturing 10, 115–126.CrossRefGoogle Scholar
Brown, D.C., & Bansal, R. (1991). Using design history systems for technology transfer. In Computer Aided Cooperative Product Development, Lecture Notes in Computer Science (Sriram, D., Logcher, R., & Fukuda, S., Eds.), Vol. 492, pp. 544–559. New York: Springer–Verlag.Google Scholar
Buckingham Shum, S., & Hammond, N. (1994). Argumentation-based design rationale: what use at what cost? International Journal of Human–Computer Studies 40(4), 603–652.CrossRefGoogle Scholar
Buckingham Shum, S., Selvin, A., Sierhuis, M., Conklin, J., Haley, C., & Nuseibeh, B. (2006). Hypermedia support for argumentation-based rationale: 15 years on from gIBIS and QOC. In Rationale Management in Software Engineering (Dutoit, A., McCall, R., Mistrík, I., & Paech, B., Eds.), pp. 111–132. Heidelberg: Springer–Verlag.CrossRefGoogle Scholar
Burge, J. (2006). Anatomy of an experiment: difficulties in evaluating rationale-based systems. Workshop on Design Rationale: Problems and Progress, DCC'06, Conf. Design Computing and Cognition, Eindhoven.Google Scholar
Burge, J., & Brown, D.C. (2004). An integrated approach for software design checking using rationale. In Proc. Design Computing and Cognition ‘04, pp. 557–576. Dordrecht: Kluwer Academic.CrossRefGoogle Scholar
Burge, J., & Brown, D.C. (2006). Rationale-based support for software maintenance. In Rationale Management in Software Engineering (Dutoit, A., McCall, R., Mistrík, I., & Paech, B., Eds.), pp. 273–296. Heidelberg: Springer–Verlag.CrossRefGoogle Scholar
Burge, J.E., Carroll, J.M., McCall, R., & Mistrík, I. (2008). Rationale-Based Software Engineering. Heidelberg: Springer–Verlag.CrossRefGoogle Scholar
Carroll, J., & Rosson, M. (1992). Getting around the task-artifact cycle: how to make claims and design by scenario. ACM Transactions on Information Systems 10(2), 181–212.CrossRefGoogle Scholar
Carroll, J., & Rosson, M. (2003). Design rationale as theory. In HCI Models, Theories, and Frameworks: Toward a Multidisciplinary Science (Carroll, J.M., Ed.), pp. 431–461. San Francisco, CA: Morgan–Kaufmann.CrossRefGoogle Scholar
Carroll, J.M. (2000). Making Use: Scenario-Based Design of Human–Computer Interactions. Cambridge, MA: MIT Press.Google Scholar
Chung, P.W.H., & Goodwin, R. (1998). An integrated approach to representing and accessing design rationale. Engineering Applications of Artificial Intelligence 11(1), 149–159.CrossRefGoogle Scholar
CMMI Product Team. (2006). CMMI for Development. Version 1.2, CMU/SEI-2006-TR-008.Google Scholar
Conklin, E.J., & Burgess-Yakemovic, K.C. (1996). A process-oriented approach to design rationale. In Design Rationale Concepts, Techniques, and Use (Moran, T., & Carroll, J., Eds.), pp. 393–427. Hillsdale, NJ: Erlbaum.Google Scholar
Curtis, B. (2000). From MCC and CMM: technology transfers bright and dim. Proc. 22nd ICSE, pp. 521–530.CrossRefGoogle Scholar
de la Garza, J.M., & Alcantara, P.T. (1997). Using parameter dependency network to represent design rationale. Journal of Computing in Civil Engineering 11(2), 102–112.CrossRefGoogle Scholar
Dellen, B., Kohler, K., & Maurer, F. (1996). Integrating software process models and design rationales. Proc. Knowledge-Based Software Engineering, pp. 84–93.CrossRefGoogle Scholar
De Medeiros, A.P., Schwabe, D., & Feijo, B. (2005). Kuaba ontology: design rationale representation and re-use in model-based designs. Conceptual Modeling—ER 2005, 24th Int. Conf. Conceptual Modeling, pp. 241–255.CrossRefGoogle Scholar
Dutoit, A., McCall, R., Mistrík, I., & Paech, B., Eds. (2006). Rationale Management in Software Engineering. Heidelberg: Springer–Verlag.Google Scholar
Dutoit, A.H., Wolf, T., Paech, B., Borner, L., & Ruckert, J. (2005). Using rationale for software engineering education. Proc. 18th Conf. Software Engineering Education and Training, pp. 129–136.CrossRefGoogle Scholar
Erdogmus, H., Favaro, J., & Halling, M. (2006). Valuation of software initiatives under uncertainty: concepts, issues, and techniques. In Value-Based Software Engineering (Biffl, S., Aurum, A., Boehm, B., Erdogmus, H., & Grünbacher, P., Eds.), pp. 39–99. Heidelberg: Springer–Verlag.CrossRefGoogle Scholar
Falessi, D., Cantone, G., & Becker, M. (2006). Documenting design decision rationale to improve individual and team design decision making: an experimental evaluation. Proc. ISESE, pp. 134–143.CrossRefGoogle Scholar
Fischer, G., Lemke, A., McCall, R., & Morch, A. (1996). Making argumentation serve design. In Design Rationale Concepts, Techniques, and Use (Moran, T., & Carroll, J., Eds.), pp. 267–293. Hillsdale, NJ: Erlbaum.Google Scholar
Garcia, A., Howard, H., & Stefik, M. (1993). Active Design Documents: A New Approach for Supporting Documentation in Preliminary Routine Design, Technical Report 82. Stanford, CA: Stanford University.Google Scholar
Garcia, A., Howard, H., & Stefik, M. (1994). Design Rationale for Collaboration: The Active Document Approach, Technical Report MCC08/94. Rio de Janeiro: Universidade Católica do Rio de Janeiro, Departmento de Informática Pontifícia.Google Scholar
Garcia, A.C.B., & Vivacqua, A.S. (1997). MultiADD: Multiagent Active Design Documents. Accessed at http://citeseer.ist.psu.edu/242337.html on April 14, 2007.Google Scholar
Gill, S., & Munson, E. (2006). A version-aware tool for design rationale. Proc. WebMedia'06, pp. 20–26.CrossRefGoogle Scholar
Gruber, T. (1990). Model-based explanation of design rationale. Proc. AAAI-90 Explanation Workshop.Google Scholar
Grudin, J. (1996). Evaluating opportunities for design capture. In Design Rationale Concepts, Techniques, and Use (Moran, T., & Carroll, J., Eds.), pp. 454–470. Hillsdale, NJ: Erlbaum.Google Scholar
Hagge, L., Houdek, F., Lappe, K., & Paech, B. (2006). Using patterns for sharing requirements engineering process rationales. In Rationale Management in Software Engineering (Dutoit, A., McCall, R., Mistrík, I., & Paech, B., Eds.), pp. 409–427. Heidelberg: Springer–Verlag.CrossRefGoogle Scholar
Hartmann, D. (2006). Interview: Jim Johnson of the Standish Group, InfoQ. Accessed at http://www.infoq.com/articles/Interview-Johnson-Standish-CHAOS on April 30, 2007.Google Scholar
Haynes, S.R. (2006). Three studies of design rationale as explanation. In Rationale Management in Software Engineering (Dutoit, A., McCall, R., Mistrík, I., & Paech, B., Eds.), pp. 53–71. Heidelberg: Springer–Verlag.CrossRefGoogle Scholar
Heliades, G., & Edmonds, E. (1999). On facilitating knowledge transfer in software design. Knowledge-Based Systems 12, 391–395.CrossRefGoogle Scholar
Hordijk, W., & Wieringa, R. (2006). Reusable rationale blocks: improving quality and efficiency of design choices. In Rationale Management in Software Engineering (Dutoit, A., McCall, R., Mistrík, I., & Paech, B., Eds.), pp. 353–371. Heidelberg: Springer–Verlag.CrossRefGoogle Scholar
Horner, J., & Attwood, M.E. (2006). Effective design rationale: understanding the barriers. In Rationale Management in Software Engineering (Dutoit, A., McCall, R., Mistrík, I., & Paech, B., Eds.), pp. 73–90. Heidelberg: Springer–Verlag.CrossRefGoogle Scholar
In, H., Boehm, B., Rodgers, T., & Deutsch, M. (2001). Applying WinWin to quality requirements: a case study. Proc. 23rd Int. Conf. Software Engineering, pp. 555–564.Google Scholar
Karacapilidis, N., & Papadias, D. (2001). Computer supported argumentation and collaborative decision making: the HERMES system. Information Systems 26(4), 259–277.CrossRefGoogle Scholar
Karsenty, L. (1996). An empirical evaluation of design rationale documents. Proc. SIGCHI Conf. Human Factors in Computing Systems: Common Ground, pp. 150–156.CrossRefGoogle Scholar
Keith, S., Blandford, A., Fields, R. & Theng, Y.L. (2002). An investigation into the application of Claims Analysis to evaluate usability of a digital library interface. Proc. JCDL Workshop on Usability.Google Scholar
Klein, M. (1992). DRCS: an integrated system for capture of designs and their rationale. Proc. Artificial Intelligence in Design ’92 (Gero, J., Ed.), pp. 393–412. Dordrect: Kluwer Academic.CrossRefGoogle Scholar
Klein, M. (1997 a). An exception handling approach to enhancing consistency, completeness and correctness in collaborative requirements capture. Concurrent Engineering Research and Applications, pp. 37–46.CrossRefGoogle Scholar
Klein, M. (1997 b). Capturing geometry rationale for collaborative design. Proc. 6th Int. Workshop on Enabling Technologies: Infrastructure for Collaborative Enterprises.Google Scholar
Kunz, W., & Rittel, H. (1970). Issues as Elements of Information Systems, Working Paper 131. Berkeley, CA: University of California, Berkeley, Center for Urban and Regional Development.Google Scholar
Lacaze, X., Palanque, P., Barboni, E., Bastide, R., & Navarre, D. (2006). From DREAM to reality: specificities of interactive systems development with respect to rationale management. In Rationale Management in Software Engineering (Dutoit, A., McCall, R., Mistrík, I., & Paech, B., Eds.), pp. 155–172. Heidelberg: Springer–Verlag.CrossRefGoogle Scholar
Lambell, N.J., Ball, L.J., & Ormerod, T.C. (2000). The evaluation of Desperado: a computerised tool to aid design reuse. In People and Computers XVI: Usability or Else! (McDonald, S., Wearn, Y., & Cockton, G., Eds.), pp. 437–453. Cambridge: Cambridge University Press.CrossRefGoogle Scholar
Larsson, M., Wall, A., Norström, C., & Crnkovic, I. (2006). Technology transfer: why some succeed and some don't. Proc. Int. Workshop on Software Technology Transfer in Software Engineering, pp. 23–28.CrossRefGoogle Scholar
Lee, J. (1991). SIBYL: a qualitative design management system. In Artificial Intelligence at MIT: Expanding Frontiers (Winston, P.H., & Shellard, S., Eds.), pp. 104–133. Cambridge MA: MIT Press.Google Scholar
Liebowitz, J. (2002). A look at NASA Goddard Space Flight Center's knowledge management initiatives. IEEE Software 19(3), 40–42.CrossRefGoogle Scholar
Lougher, R., & Rodden, T. (1993). Group support for the recording and sharing of maintenance rationale. Software Engineering Journal 8(6), 295–306.CrossRefGoogle Scholar
Louridas, P., & Loucopoulos, P. (2000). A generic model for reflective design. ACM Transactions on Software Engineering Methodology 9(2), 199–237.CrossRefGoogle Scholar
MacLean, A., Young, R.M., Bellotti, V., & Moran, T.P. (1996). Questions, options and criteria: elements of design space analysis. In Design Rationale Concepts, Techniques, and Use (Moran, T., & Carroll, J., Eds.), pp. 201–251. Hillsdale, NJ: Erlbaum.Google Scholar
McGill, K., Deadrick, W., Hayes, J.H., & Dekhtyar, A. (2006). Houston, we have a success story: technology transfer at the NASA IV&V facility. Proc. Int. Workshop on Software Technology Transfer in Software Engineering, pp. 49–54.CrossRefGoogle Scholar
Moran, T., & Carroll, J., Eds. (1996). Design Rationale Concepts, Techniques, and Use. Hillsdale, NJ: Erlbaum.Google Scholar
Myers, K., Zumel, N., & Garcia, P. (1999). Automated capture of rationale for the detailed design process. Proc. 11th National Conf. Innovative Applications of Artificial Intelligence, pp. 876–883.Google Scholar
Palyagar, B., & Richards, D. (2006). Capturing and reusing rationale associated with requirements engineering process improvement: a case study. In Rationale Management in Software Engineering (Dutoit, A., McCall, R., Mistrík, I., & Paech, B., Eds.), pp. 391–408. Heidelberg: Springer–Verlag.CrossRefGoogle Scholar
Peña-Mora, F., Sriram, D., & Logcher, R. (1995). Design rationale for computer-supported conflict mitigation. ASCE Journal of Computing in Civil Engineering 9(1), 57–72.CrossRefGoogle Scholar
Peña-Mora, F., & Vadhavkar, S. (1997). Augmenting design patterns with design rationale. Artificial Intelligence for Engineering Design, Analysis and Manufacturing 11, 93–108.CrossRefGoogle Scholar
Potts, C., Takahashi, K., & Anton, A. (1994). Inquiry-based requirements analysis. IEEE Software 11(2), 21–32.CrossRefGoogle Scholar
Potts, C., Takahashi, K., Smith, J., & Ota, K. (1995). An evaluation of inquiry-based requirements analysis for an Internet service. Proc. 2nd IEEE Int. Symp. Requirements Engineering, pp. 172–180.CrossRefGoogle Scholar
Ramesh, B., & Dhar, V. (1994). Representing and maintaining process knowledge for large-scale systems development. IEEE Expert: Intelligent Systems and Their Applications 9(2), 54–59.CrossRefGoogle Scholar
Ramesh, B., & Dhar, V. (1992). Supporting systems development by capturing deliberations during requirements engineering. IEEE Transactions on Software Engineering 18(6), 498–510.CrossRefGoogle Scholar
Redwine, S., & Riddle, W. (1985). Software technology maturation. Proc. 8th ICSE, pp. 189–2000.Google Scholar
Richter, H., & Abowd, G. (1999). Automating The Capture of Design Knowledge: A Preliminary Study, Technical Report GVU-99-45. Atlanta, GA: Georgia Tech.Google Scholar
Richter, H., Schuchhard, P., & Abowd, G. (1998). Automated Capture and Retrieval of Architectural Rationale, Technical Report GIT-GVU-98-37. Atlanta, GA: Georgia Tech.Google Scholar
Rus, I., & Lindvall, M. (2002). Guest editors’ introduction: knowledge management in software engineering. IEEE Software 19(3), 26–38.CrossRefGoogle Scholar
Schneider, K. (2006). Rationale as a by-product, In Rationale Management in Software Engineering (Dutoit, A., McCall, R., Mistrík, I., & Paech, B., Eds.), pp. 91–109. Heidelberg: Springer–Verlag.CrossRefGoogle Scholar
SEI. (1997). Integrated Product Development Capability Maturity Model, Draft Version 0.98. Pittsburgh, PA: Carnegie Mellon University, Enterprise Process Improvement Collaboration and Software Engineering Institute.Google Scholar
Selvin, A., & Sierhuis, M. (1999). Case studies of project Compendium in different organizations. Workshop on Computer-Supported Collaborative Argumentation Conf. Computer-Supported Collaborative Learning.Google Scholar
Shah, J., Rangaswamy, S., Qureshi, S., & Urban, S. (1999). Design history system: data models and prototype implementation. In Knowledge Intensive CAD (Tomiyama, T., Mäntylä, M., & Finger, S., Eds.), pp. 91–114. Dordrecht: Kluwer Academic.Google Scholar
Shipman, F., & McCall, R. (1997). Integrating different perspectives on design rationale: supporting the emergence of design rationale from design communication. Artificial Intelligence for Engineering Design, Analysis and Manufacturing 11, 141–154.CrossRefGoogle Scholar
Smith, R., & Farquhar, A. (2000). The road ahead for knowledge management: an AI perspective. AI Magazine 21(4), 17–40.Google Scholar
Standish Group. (1994). CHAOS Report. Accessed at http://www.standishgroup.com/sample_research/chaos_1994_1.php on April 6, 2007.Google Scholar
Tang, A., Babar, M., Gorton, I., & Han, J. (2006). A survey of architecture design rationale. Journal of Systems and Software 79, 1792–1804.CrossRefGoogle Scholar
Tang, A., Jin, Y., & Han, J. (2007). A rationale-based architecture model for design traceability and reasoning. Journal of Systems and Software 80(6), 918–934.CrossRefGoogle Scholar
Taura, T., & Kubota, A. (1999). A study on engineering history base. Research in Engineering Design 11(1), 45–54.CrossRefGoogle Scholar
van der Ven, J.S., Jansen, A.G.J, Nijhuis, J.A., & Bosch, J. (2006). Design decisions: the bridge between rationale and architecture. In Rationale Management in Software Engineering (Dutoit, A., McCall, R., Mistrík, I., & Paech, B., Eds.), pp. 329–348. Heidelberg: Springer–Verlag.Google Scholar
Vetschera, R. (2006). Preference-based decision support in software engineering. In Value-Based Software Engineering (Biffl, S., Aurum, A., Boehm, B., Erdogmus, H., & Grünbacher, P., Eds.), pp. 67–89. Heidelberg: Springer–Verlag.CrossRefGoogle Scholar
Zaychik, V., & Regli, W.C. (2003). Capturing communication and context in the software project lifecycle. Research in Engineering Design 14(2), 75–88.CrossRefGoogle Scholar