Hostname: page-component-586b7cd67f-t7czq Total loading time: 0 Render date: 2024-11-25T21:49:58.876Z Has data issue: false hasContentIssue false
  • English
  • Français

Follow-up question handling in the IMIX and Ritel systems: A comparative study

Published online by Cambridge University Press:  01 January 2009

B. W. VAN SCHOOTEN ,
R. OP DEN AKKER ,
S. ROSSET ,
O. GALIBERT ,
A. MAX  and
G. ILLOUZ
B. W. VAN SCHOOTEN
Affiliation:
Human Media Interaction, University of Twente, Netherlands e-mail: [email protected], [email protected]
R. OP DEN AKKER
Affiliation:
Human Media Interaction, University of Twente, Netherlands e-mail: [email protected], [email protected]
S. ROSSET
Affiliation:
Spoken Language Processing Group (TLP), CNRS-LIMSI, France e-mail: [email protected], [email protected], [email protected], [email protected]
O. GALIBERT
Affiliation:
Spoken Language Processing Group (TLP), CNRS-LIMSI, France e-mail: [email protected], [email protected], [email protected], [email protected]
A. MAX
Affiliation:
Spoken Language Processing Group (TLP), CNRS-LIMSI, France e-mail: [email protected], [email protected], [email protected], [email protected]
G. ILLOUZ
Affiliation:
Spoken Language Processing Group (TLP), CNRS-LIMSI, France e-mail: [email protected], [email protected], [email protected], [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

One of the basic topics of question answering (QA) dialogue systems is how follow-up questions should be interpreted by a QA system. In this paper, we shall discuss our experience with the IMIX and Ritel systems, for both of which a follow-up question handling scheme has been developed, and corpora have been collected. These two systems are each other's opposites in many respects: IMIX is multimodal, non-factoid, black-box QA, while Ritel is speech, factoid, keyword-based QA. Nevertheless, we will show that they are quite comparable, and that it is fruitful to examine the similarities and differences. We shall look at how the systems are composed, and how real, non-expert, users interact with the systems. We shall also provide comparisons with systems from the literature where possible, and indicate where open issues lie and in what areas existing systems may be improved. We conclude that most systems have a common architecture with a set of common subtasks, in particular detecting follow-up questions and finding referents for them. We characterise these tasks using the typical techniques used for performing them, and data from our corpora. We also identify a special type of follow-up question, the discourse question, which is asked when the user is trying to understand an answer, and propose some basic methods for handling it.

Type
Papers
Information
Natural Language Engineering , Volume 15 , Special Issue 1: Interactive Question Answering , January 2009 , pp. 97 - 118
Copyright
Copyright © Cambridge University Press 2008

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Bertomeu, N., Uszkoreit, H., Frank, A., Krieger, H.-U., and Jörg, B. 2006. Contextual phenomena and thematic relations in database qa dialogues: results from a wizard-of-oz experiment. In Workshop on Interactive Question Answering, HLT-NAACL 06, New York, USA.CrossRefGoogle Scholar
Boves, L., and den Os, E. 2005. Interactivity and multimodality in the imix demonstrator. In Multimedia and Expo, ICME 2005, Amsterdam, pp. 1578–1581, IEEE, NJ, USA.Google Scholar
De Boni, M., and Manandhar, S. 2004. Implementing clarification dialogues in open domain question answering. Journal of Natural Language Engineering 11 4343–361CrossRefGoogle Scholar
Dix, J., Finlay, J., Abowd, G., and Beale, R. 2004. Human–Computer Interaction, 3rd ed.Harlow: Pearson/Prentice Hall.Google Scholar
Fukumoto, J., Niwa, T., Itoigawa, M., and Matsuda, M. 2004. RitsQA: list answer detection and context task with ellipses handling. In Working notes of the Fourth NTCIR Workshop Meeting, Tokyo, Japan, pp. 310–314.Google Scholar
Futrelle, R. P., and Rumshisky, A. 2001. Discourse structure of text-graphics documents. In First International Symposium on Smart Graphics, New York, USA, pp. 31–38, Heidelberg: Springer.Google Scholar
Galibert, O., Illouz, G., and Rosset, S. 2005. Ritel: an open-domain, human–computer dialog system. In Interspeech 2005, Lisbon, Portugal, pp. 909–912, Bonn: ISCA.CrossRefGoogle Scholar
Hickl, A., Wang, P., Lehmann, J., and Harabagiu, S. M. 2006. FERRET: interactive question-answering for real-world environments. In ACL 2006, Sydney, Australia.Google Scholar
Inui, K., Yamashita, A., and Matsumoto, Y. 2003. Dialogue management for language-based information seeking. In Proceedings of the First International Workshop on Language Understanding and Agents for Real World Interaction, Sapporo, Japan, pp. 32–38.Google Scholar
Kato, T., Fukumoto, J., and Masui, F. 2004. Question answering challenge for information access dialogue – overview of NTCIR4 QAC2 subtask 3. In Working notes of the Fourth NTCIR Workshop Meeting, Tokyo, Japan.Google Scholar
Kato, T., Fukumoto, J., Masui, F., and Kando, N. 2006. Woz simulation of interactive question answering. In Workshop on Interactive Question Answering, HLT-NAACL 06, New York, USA.CrossRefGoogle Scholar
Lappin, S., and Leass, H. J. 1994. An algorithm for pronominal anaphora resolution. Computational Linguistics 20 4535–561Google Scholar
Lin, C.-J., and Chen, H.-H. 2001. Description of NTU system at TREC-10 QA track. In TREC 10, Gaithersburg, MD, USA.Google Scholar
Lin, J., Quan, D., Sinha, V., Bakshi, K., Huynh, D., Katz, B., and Karger, D. R. 2003. What makes a good answer? The role of context in question answering. In Proceedings of the Ninth IFIP TC13 International Conference on Human–Computer Interaction (INTERACT-2003), Zurich, Switzerland.CrossRefGoogle Scholar
Martin, J.-C., Buisine, S., Pitel, G., and Bernsen, N. O. 2006. Fusion of children's speech and 2D gestures when conversing with 3D characters. Signal Processing Journal 86 (12): 3596–3624 (special issue on multimodal interfaces).CrossRefGoogle Scholar
Moore, J. D. 1989. Responding to ‘Huh?’: answering vaguely-articulated follow-up questions. In Proceedings of the Conference on Human Factors in Computing Systems, Austin, TX, USA.CrossRefGoogle Scholar
Oh, J.-H., Lee, K.-S., Chang, D.-S., Seo, C. W., and Choi, K.-S. 2001. Trec-10 experiments at kaist: batch filtering and question answering. In TREC, Gaithersburg, MD, USA.Google Scholar
Reithinger, N., Bergweiler, S., Engel, R., Herzog, G., Pfleger, N., Romanelli, M., and Sonntag, D. 2005. A look under the hood: design and development of the first smartweb system demonstrator. In ICMI '05: Proceedings of the Seventh International Conference on Multimodal Interfaces, pp. 159–166. New York, NY: ACM Press.CrossRefGoogle Scholar
Rosset, S., and Petel, S. 2006. The Ritel corpus – an annotated human–machine open-domain question answering spoken dialog corpus. In International Conference on Language Resources and Evaluation, Genoa, Italy.Google Scholar
Russell, B. C., Torralba, A., Murphy, K. P., and Freeman, W. T. 2005. LabelMe: a database and web-based tool for image annotation. Technical report, MIT, MIT AI Lab Memo AIM-2005-025.Google Scholar
Small, S., Liu, T., Shimizu, N., and Strzalkowski, T. 2003. HITIQA: an interactive question answering system: a preliminary report. In Proceedings of the ACL 2003 Workshop on Multilingual Summarization and Question Answering, Sapporo, Japan.CrossRefGoogle Scholar
Theune, M., Krahmer, E., van Schooten, B., op den Akker, R., van Hooijdonk, C., Marsi, E., Bosma, W., Hofs, D., and Nijholt, A. 2007. Questions, pictures, answers: introducing pictures in question-answering systems. In Tenth international symposium on social communication, Universidad de Oriente Santiago de Cuba, pp. 469–474.Google Scholar
van Schooten, B., and op den Akker, R. 2005. Follow-up proceedings of ntcir-5 workshop meeting, December 6–9, 2005, Tokyo, Japanutterances in QA dialogue. Traitement Automatique des Langues, 46(3): 181–206.Google Scholar
van Schooten, B., and op den Akker, R. 2007. Multimodal follow-up questions to multimodal answers in a QA system. In Tenth International Symposium on Social Communication, Universidad de Oriente Santiago de Cuba, pp. 469–474.Google Scholar
van Schooten, B., Rosset, S., Galibert, O., Max, A., op den Akker, R., and Illouz, G. 2007. Handling speech input in the Ritel QA dialogue system. In Interspeech 2007, pp. 181–206.Google Scholar
Voorhees, E. 2005. Overview of the TREC 2005 question answering track. Technical report, NIST.CrossRefGoogle Scholar
Willems, D. J. M., Rossignol, S. Y. P., and Vuurpijl, L. G. 2005. Features for mode detection in natural online pen input. In BIGS 2005: Proceedings of the Twelfth Biennial Conference of the International Graphonomics Society, Salerno, Italy, pp. 113–117, Civitella: Editrice Zona.Google Scholar
Yang, F., Feng, J., and Di Fabbrizio, G. 2006. A data driven approach to relevancy recognition for contextual question answering. In Workshop on Interactive Question Answering, HLT-NAACL 06, New York, USA.CrossRefGoogle Scholar