Hostname: page-component-78c5997874-4rdpn Total loading time: 0 Render date: 2024-11-08T05:10:59.784Z Has data issue: false hasContentIssue false
  • English
  • Français

Does this list contain what you were searching for? Learning adaptive dialogue strategies for interactive question answering

Published online by Cambridge University Press:  01 January 2009

V. RIESER  and
O. LEMON
V. RIESER
Affiliation:
School of Informatics, University of Edinburgh, Edinburgh, EH8 9AB, Great Britain e-mail: [email protected], [email protected]
O. LEMON
Affiliation:
School of Informatics, University of Edinburgh, Edinburgh, EH8 9AB, Great Britain e-mail: [email protected], [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

Policy learning is an active topic in dialogue systems research, but it has not been explored in relation to interactive question answering (IQA). We take a first step in learning adaptive interaction policies for question answering : we address the question of how to acquire enough reliable query constraints, how many database results to present to the user and when to present them, given the competing trade-offs between the length of the answer list, the length of the interaction, the type of database and the noise in the communication channel. The operating conditions are reflected in an objective function which we use to derive a hand-coded threshold-based policy and rewards to train a reinforcement learning policy. The same objective function is used for evaluation. We show that we can learn strategies for this complex trade-off problem which perform significantly better than a variety of hand-coded policies, for a wide range of noise conditions, user types, types of DB and turn-penalties. Our policy learning framework thus covers a wide spectrum of operating conditions. The learned policies produce an average relative increase in reward of 86.78% over the hand-coded policies. In 93% of the cases the learned policies perform significantly better than the hand-coded ones (p < .001). Furthermore we show that the type of database has a significant effect on learning and we give qualitative descriptions of the learned IQA policies.

Type
Papers
Information
Natural Language Engineering , Volume 15 , Special Issue 1: Interactive Question Answering , January 2009 , pp. 55 - 72
Copyright
Copyright © Cambridge University Press 2008

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Androutsopoulos, I., and Ritchie, G. 2000. Database interfaces. In Dale, R., Moisl, H., and Somers, H. (eds.), Handbook of Natural Language Processing, pp. 209–40. New York, NY: Marcel Dekker.Google Scholar
Becker, T., Blaylock, N., Gerstenberger, C., Korthauer, A., Perera, N., Pitz, M., Poller, P., Schehl, J., Steffens, F., Stegmann, R., and Steigner, J. 2007. In-car showcase based on TALK libraries. Technical Report, Deliverable 5.3, TALK Project.Google Scholar
Chung, G. 2004. Developing a flexible spoken dialog system using simulation. In Proceedings of Association for Computational Linguistics (ACL), Barcelona, Spain.CrossRefGoogle Scholar
Demberg, V., and Moore, J. 2006. Information presentation in spoken dialogue systems. In Proceedings of European Association for Computational Linguistics (EACL), Trento, Italy.Google Scholar
Divi, V., Forlines, C., Gemert, J., van Raj, B., Schmidt-Nielsen, B., Wittenburg, K., Woelfel, J., Wolf, P., and Zhang, F. F. 2004. A speech-in list-out approach to spoken user interfaces. In Proceedings of Human Language Technologies Conference (HLT), Boston, MA, USA.CrossRefGoogle Scholar
Dohsaka, K., Yasuda, N., and Aikawa, K. 2003. Efficient spoken dialogue control depending on the speech recognition rate and system's database. In Proceedings of Eurospeech, Geneva, Switzerland.CrossRefGoogle Scholar
Georgila, K., Henderson, J., and Lemon, O. 2006. User simulation for spoken dialogue systems: learning and evaluation. In Proceedings of Interspeech/ICSLP 2006, Pittsburgh, PA, USA.Google Scholar
Lemon, O., Georgila, K., and Henderson, J. 2006. Evaluating effectiveness and portability of reinforcement learned dialogue strategies with real users: the TALK TownInfo evaluation. In Proceedings of Spoken Language Technology (SLT), Palm Beach, Aruba.Google Scholar
Lemon, O., and Liu, X. 2007. Dialogue policy learning for combinations of noise and user simulations: transfer results. In Proceedings of 8th SIGdial Workshop, Association for Computational Linguistics, Antwerp, Belgium, pp. 55–58.Google Scholar
Lemon, O., Liu, X., Shapiro, D., and Tollander, C. 2006. Hierarchical reinforcement learning of dialogue policies in a development environment for dialogue systems: REALL-DUDE. In Proceedings of 10th SEMdial Workshop on the Semantics and Pragmatics of Dialogue (BRANDIAL), Potsdam, Germany.Google Scholar
Levin, E., and Pieraccini, R. 1997. A stochastic model of computer-human interaction for learning dialogue strategies. In Proceedings of Eurospeech, Rhodos, Greece.CrossRefGoogle Scholar
Levin, E., Pieraccini, R., and Eckert, W. 2000. A stochastic model of human-machine interaction for learning dialog strategies. IEEE Transactions on Speech and Audio Processing 8 (1): 11–23.Google Scholar
Oviatt, S., Levow, G.-A., MacEarchern, M., and Kuhn, K. 1996. Modelling hyperarticulate speech during human-computer error resolution. In Proceedings of the International Conference on Spoken Language Processing (ICSLP), Philadelphia, PA, USA.CrossRefGoogle Scholar
Paek, T. 2006. Reinforcement learning for spoken dialogue systems: comparing strengths and weaknesses for practical deployment. In Dialogue on Dialogues. Interspeech/ICSLP Satellite Workshop, Pittsburgh, PA, USA.Google Scholar
Paek, T., and Horvitz, E. 2000. Conversation as action under uncertainty. In Proceedings of the 16th Conference on Uncertainty in Artificial Intelligence (UAI), Stanford, CA, USA.Google Scholar
Pietquin, O. 2006. Machine learning for spoken dialogue management: an experiment with speech-based database querying. In Euzenat, J., and Domingue, J. (ed.), Artificial Intelligence: Methodology, Systems & Applications. Lecture Notes in Artificial Intelligence, vol. 4183, pp. 172–80. Berlin: Springer Verlag.CrossRefGoogle Scholar
Pietquin, O., and Dutoit, T. 2006. A probabilistic framework for dialog simulation and optimal strategy learning. IEEE Transactions on Audio, Speech and Language Processing 14 (2): 589–99.Google Scholar
Rieser, V., Kruijff-Korbayová, I., and Lemon, O. 2005. A corpus collection and annotation framework for learning multimodal clarification strategies. In Proceedings of 6th SIGdial Workshop, Lisbon, Portugal.Google Scholar
Rieser, V., and Lemon, O. 2006a. Cluster-based user simulations for learning dialogue strategies. In Proceedings of Interspeech/ICSLP 2006, Pittsburgh, PA, USA.CrossRefGoogle Scholar
Rieser, V., and Lemon, O. 2006b. Using machine learning to explore human multimodal clarification strategies. In Proceedings of Association for Computational Linguistics (ACL), Sydney, Australia.CrossRefGoogle Scholar
Scheffler, K., and Young, S. J. 2002. Automatic learning of dialogue strategy using dialogue simulation and reinforcement learning. In Proceedings of Human Language Technology (HLT), San Diego, CA, USA.Google Scholar
Shapiro, D., and Langley, P. 2002. Separating skills from preference: using learning to program by reward. In Proceedings of 19th International Conference on Machine Learning, Sydney, Australia.Google Scholar
Singh, S., Litman, D., Kearns, M., and Walker, M. 2002. Optimizing dialogue management with reinforcement learning: experiments with the NJFun system. Journal of Artificial Intelligence Research (JAIR) 16: 105–33.Google Scholar
Skantze, G. 2007. Making grounding decisions: data-driven estimation of dialogue costs and confidence thresholds. In Proceedings of 8th SIGdial, Antwerp, Belgium.Google Scholar
Strzalkowski, T., Small, S., Hardy, H., Kantor, P., Min, W., Ryan, S., Shimizu, N., Ting, L., Wacholder, N., and Yamrom, B. 2008. Question answering as dialogue with data. In Strzalkowski, T., and Harabagiu, S. (ed.), Advances in Open Domain Question Answering. Text, Speech and Language Technology, vol. 32. Springer-Verlag, New York, NY, USA.CrossRefGoogle Scholar
Sutton, R., and Barto, A. 1998. Reinforcement Learning. MIT Press, Cambridge, MA, USA.Google Scholar
Varges, S., Weng, F., and Pon-Barry, H. 2006. Interactive question answering and constraint relaxation in spoken dialogue systems. In Proceedings of 7th SIGdial Workshop, Sydney, Australia.Google Scholar
Walker, M. 2005. Can we talk? Methods for evaluation and training of spoken dialogue system. Language Resources and Evaluation 39 (1): 65–75.Google Scholar
Walker, M. A., Kamm, C. A., and Litman, D. J. 2000. Towards developing general models of usability with PARADISE. Natural Language Engineering 6 (3–4): 363–77.CrossRefGoogle Scholar
Walker, M. A., Passonneau, R. J., and Boland, J. E. 2001. Quantitative and qualitative evaluation of DARPA communicator spoken dialogue systems. In Proceedings of Association for Computational Linguistics (ACL), Toulouse, France.CrossRefGoogle Scholar
Zhao, X. 2007. Integrating a QA System with Dialogue Management for the Music Domain. Master's Thesis, School of Informatics, University of Edinburgh.Google Scholar