Skip to main content

Follow-up question handling in the IMIX and Ritel systems: A comparative study

  • B. W. VAN SCHOOTEN (a1), R. OP DEN AKKER (a1), S. ROSSET (a2), O. GALIBERT (a2), A. MAX (a2) and G. ILLOUZ (a2)...

One of the basic topics of question answering (QA) dialogue systems is how follow-up questions should be interpreted by a QA system. In this paper, we shall discuss our experience with the IMIX and Ritel systems, for both of which a follow-up question handling scheme has been developed, and corpora have been collected. These two systems are each other's opposites in many respects: IMIX is multimodal, non-factoid, black-box QA, while Ritel is speech, factoid, keyword-based QA. Nevertheless, we will show that they are quite comparable, and that it is fruitful to examine the similarities and differences. We shall look at how the systems are composed, and how real, non-expert, users interact with the systems. We shall also provide comparisons with systems from the literature where possible, and indicate where open issues lie and in what areas existing systems may be improved. We conclude that most systems have a common architecture with a set of common subtasks, in particular detecting follow-up questions and finding referents for them. We characterise these tasks using the typical techniques used for performing them, and data from our corpora. We also identify a special type of follow-up question, the discourse question, which is asked when the user is trying to understand an answer, and propose some basic methods for handling it.

Hide All
Bertomeu, N., Uszkoreit, H., Frank, A., Krieger, H.-U., and Jörg, B. 2006. Contextual phenomena and thematic relations in database qa dialogues: results from a wizard-of-oz experiment. In Workshop on Interactive Question Answering, HLT-NAACL 06, New York, USA.
Boves, L., and den Os, E. 2005. Interactivity and multimodality in the imix demonstrator. In Multimedia and Expo, ICME 2005, Amsterdam, pp. 15781581, IEEE, NJ, USA.
De Boni, M., and Manandhar, S. 2004. Implementing clarification dialogues in open domain question answering. Journal of Natural Language Engineering 11 4343361
Dix, J., Finlay, J., Abowd, G., and Beale, R. 2004. Human–Computer Interaction, 3rd ed.Harlow: Pearson/Prentice Hall.
Fukumoto, J., Niwa, T., Itoigawa, M., and Matsuda, M. 2004. RitsQA: list answer detection and context task with ellipses handling. In Working notes of the Fourth NTCIR Workshop Meeting, Tokyo, Japan, pp. 310–314.
Futrelle, R. P., and Rumshisky, A. 2001. Discourse structure of text-graphics documents. In First International Symposium on Smart Graphics, New York, USA, pp. 31–38, Heidelberg: Springer.
Galibert, O., Illouz, G., and Rosset, S. 2005. Ritel: an open-domain, human–computer dialog system. In Interspeech 2005, Lisbon, Portugal, pp. 909912, Bonn: ISCA.
Hickl, A., Wang, P., Lehmann, J., and Harabagiu, S. M. 2006. FERRET: interactive question-answering for real-world environments. In ACL 2006, Sydney, Australia.
Inui, K., Yamashita, A., and Matsumoto, Y. 2003. Dialogue management for language-based information seeking. In Proceedings of the First International Workshop on Language Understanding and Agents for Real World Interaction, Sapporo, Japan, pp. 32–38.
Kato, T., Fukumoto, J., and Masui, F. 2004. Question answering challenge for information access dialogue – overview of NTCIR4 QAC2 subtask 3. In Working notes of the Fourth NTCIR Workshop Meeting, Tokyo, Japan.
Kato, T., Fukumoto, J., Masui, F., and Kando, N. 2006. Woz simulation of interactive question answering. In Workshop on Interactive Question Answering, HLT-NAACL 06, New York, USA.
Lappin, S., and Leass, H. J. 1994. An algorithm for pronominal anaphora resolution. Computational Linguistics 20 4535561
Lin, C.-J., and Chen, H.-H. 2001. Description of NTU system at TREC-10 QA track. In TREC 10, Gaithersburg, MD, USA.
Lin, J., Quan, D., Sinha, V., Bakshi, K., Huynh, D., Katz, B., and Karger, D. R. 2003. What makes a good answer? The role of context in question answering. In Proceedings of the Ninth IFIP TC13 International Conference on Human–Computer Interaction (INTERACT-2003), Zurich, Switzerland.
Martin, J.-C., Buisine, S., Pitel, G., and Bernsen, N. O. 2006. Fusion of children's speech and 2D gestures when conversing with 3D characters. Signal Processing Journal 86 (12): 35963624 (special issue on multimodal interfaces).
Moore, J. D. 1989. Responding to ‘Huh?’: answering vaguely-articulated follow-up questions. In Proceedings of the Conference on Human Factors in Computing Systems, Austin, TX, USA.
Oh, J.-H., Lee, K.-S., Chang, D.-S., Seo, C. W., and Choi, K.-S. 2001. Trec-10 experiments at kaist: batch filtering and question answering. In TREC, Gaithersburg, MD, USA.
Reithinger, N., Bergweiler, S., Engel, R., Herzog, G., Pfleger, N., Romanelli, M., and Sonntag, D. 2005. A look under the hood: design and development of the first smartweb system demonstrator. In ICMI '05: Proceedings of the Seventh International Conference on Multimodal Interfaces, pp. 159–166. New York, NY: ACM Press.
Rosset, S., and Petel, S. 2006. The Ritel corpus – an annotated human–machine open-domain question answering spoken dialog corpus. In International Conference on Language Resources and Evaluation, Genoa, Italy.
Russell, B. C., Torralba, A., Murphy, K. P., and Freeman, W. T. 2005. LabelMe: a database and web-based tool for image annotation. Technical report, MIT, MIT AI Lab Memo AIM-2005-025.
Small, S., Liu, T., Shimizu, N., and Strzalkowski, T. 2003. HITIQA: an interactive question answering system: a preliminary report. In Proceedings of the ACL 2003 Workshop on Multilingual Summarization and Question Answering, Sapporo, Japan.
Theune, M., Krahmer, E., van Schooten, B., op den Akker, R., van Hooijdonk, C., Marsi, E., Bosma, W., Hofs, D., and Nijholt, A. 2007. Questions, pictures, answers: introducing pictures in question-answering systems. In Tenth international symposium on social communication, Universidad de Oriente Santiago de Cuba, pp. 469–474.
van Schooten, B., and op den Akker, R. 2005. Follow-up proceedings of ntcir-5 workshop meeting, December 6–9, 2005, Tokyo, Japanutterances in QA dialogue. Traitement Automatique des Langues, 46(3): 181–206.
van Schooten, B., and op den Akker, R. 2007. Multimodal follow-up questions to multimodal answers in a QA system. In Tenth International Symposium on Social Communication, Universidad de Oriente Santiago de Cuba, pp. 469–474.
van Schooten, B., Rosset, S., Galibert, O., Max, A., op den Akker, R., and Illouz, G. 2007. Handling speech input in the Ritel QA dialogue system. In Interspeech 2007, pp. 181–206.
Voorhees, E. 2005. Overview of the TREC 2005 question answering track. Technical report, NIST.
Willems, D. J. M., Rossignol, S. Y. P., and Vuurpijl, L. G. 2005. Features for mode detection in natural online pen input. In BIGS 2005: Proceedings of the Twelfth Biennial Conference of the International Graphonomics Society, Salerno, Italy, pp. 113–117, Civitella: Editrice Zona.
Yang, F., Feng, J., and Di Fabbrizio, G. 2006. A data driven approach to relevancy recognition for contextual question answering. In Workshop on Interactive Question Answering, HLT-NAACL 06, New York, USA.
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

Natural Language Engineering
  • ISSN: 1351-3249
  • EISSN: 1469-8110
  • URL: /core/journals/natural-language-engineering
Please enter your name
Please enter a valid email address
Who would you like to send this to? *


Full text views

Total number of HTML views: 3
Total number of PDF views: 12 *
Loading metrics...

Abstract views

Total abstract views: 86 *
Loading metrics...

* Views captured on Cambridge Core between September 2016 - 20th March 2018. This data will be updated every 24 hours.