A joint learning approach for situated language generation

doi:10.1017/CBO9780511844492.008

8 - A joint learning approach for situated language generation

from Part III - Handling uncertainty

Published online by Cambridge University Press: 05 July 2014

Nina Dethlefs and

Heriberto Cuayáhuitl

Edited by

Amanda Stent and

Srinivas Bangalore

Show author details

Nina Dethlefs: Affiliation:
Heriot-Watt University
Heriberto Cuayáhuitl: Affiliation:
Heriot-Watt University
Amanda Stent: Affiliation:
AT&T Research, Florham Park, New Jersey
Srinivas Bangalore: Affiliation:
AT&T Research, Florham Park, New Jersey

Book contents

Get access

Summary

Introduction

Interactive systems are increasingly situated: they have knowledge about the non-linguistic context of the interaction, including aspects related to location, time, and the user (Byron and Fosler-Lussier, 2006; Kelleher et al., 2006; Stoia et al., 2006; Raux and Nakano, 2010; Garoufi and Koller, 2011; Janarthanam et al., 2012). This extra knowledge makes it possible for the Natural Language Generation (NLG) components of these systems to be more adaptive, changing their output to suit the larger context. Adaptive NLG systems for situated interaction aim to produce the most effective utterance for each user in each physical and discourse context. At each stage of the generation process (what to say or content selection, how to structure content or utterance planning, and how to express content or surface realization), the best choices depend on the physical and linguistic context, which is constantly changing. Consequently, it is key to successful interaction that adaptive NLG systems constantly monitor the physical environment, the dialogue history, and the user's preferences and behaviors. As the representations of each of these will necessarily be incomplete and error-prone, adaptive NLG systems must also be able to model uncertainty in the generation process.

A designer of adaptive NLG systems faces at least two challenges. The first challenge is to identify the set of contextual features that are relevant to decision making in a specific generation situation. The second challenge is to develop a method for selecting a (near-)optimal choice in any given situation from a set of competing ones that may initially appear as viable alternatives. Complicating these challenges is the fact that individual generation decisions are tightly interrelated, so the best decision at one stage may easily depend on others.

Information

Type: Chapter
Information: Natural Language Generation in Interactive Systems , pp. 180 - 204

DOI: https://doi.org/10.1017/CBO9780511844492.008 [Opens in a new window]

Publisher: Cambridge University Press

Print publication year: 2014

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Book purchase

Temporarily unavailable

References

Angeli, G., Liang, P., and Klein, D. (2010). A simple domain-independent probabilistic approach to generation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 502-512, Boston, MA. Association for Computational Linguistics.Google Scholar

Baum, L. E., Petrie, T., Soules, G., and Weiss, N. (1970). A maximization technique occurring in the statistical analysis of probabilistic functions of Markov chains. The Annals of Mathematical Statistics, 41(1):164-171.CrossRef Google Scholar

Belz, A. and Reiter, E. (2006). Comparing automatic and human evaluation of NLG systems. In Proceedings of the Conference of the European Chapter of the Association for Computational Linguistics (EACL), pages 313-320, Trento, Italy. Association for Computational Linguistics.Google Scholar

Byron, D. and Fosler-Lussier, E. (2006). The OSU Quake 2004 corpus of two-party situated problem-solving dialogs. Technical Report OSU-CISRC-805-TR57, Ohio State University.Google Scholar

Cuayáhuitl, H. (2009). Hierarchical Reinforcement Learning for Spoken Dialogue Systems. PhD thesis, University of Edinburgh.Google Scholar

Cuayáhuitl, H. and Dethlefs, N. (2011a). Optimizing situated dialogue management in unknown environments. In Proceedings of the International Conference on Spoken Language Processing (INTERSPEECH), pages 1009-1012, Florence, Italy. International Speech Communication Association.Google Scholar

Cuayáhuitl, H. and Dethlefs, N. (2011b). Spatially-aware dialogue control using hierarchical reinforcement learning. ACM Transactions on Speech and Language Processing, 7(3):5:1–5:26.CrossRef Google Scholar

Cuayáhuitl, FL, Korbayová, I. K., and Dethlefs, N. (2012). Hierarchical dialogue policy learning using flexible state transitions and linear function approximation. In Proceedings of the International Conference on Computational Linguistics (COLING), pages 95-102, Mumbai, India. International Committee on Computational Linguistics.Google Scholar

Cuayáhuitl, H., Renals, S., Lemon, O., and Shimodaira, H. (2005). Human-computer dialogue simulation using hidden Markov models. In Proceedings of the IEEE workshop on Automatic Speech Recognition and Understanding (ASRU), pages 290-295, San Juan, Puerto Rico. Institute of Electrical and Electronics Engineers.Google Scholar

Denecke, M., Dohsaka, K., and Nakano, M. (2004). Fast reinforcement learning of dialogue policies using stable function approximation. In Proceedings of the International Joint Conference on Natural Language Processing (IJCNLP), pages 1-11, Hainan Island, China. Association for Computational Linguistics.Google Scholar

Denis, A. (2010). Generating referring expressions with reference domain theory. In Proceedings of the International Workshop on Natural Language Generation (INLG), pages 27-36, Trim, Ireland. Association for Computational Linguistics.Google Scholar

Dethlefs, N. and Cuayáhuitl, H. (2010). Hierarchical reinforcement learning for adaptive text generation. In Proceedings of the International Workshop on Natural Language Generation (INLG), pages 37-46, Trim, Ireland. Association for Computational Linguistics.Google Scholar

Dethlefs, N. and Cuayáhuitl, H. (2011a). Combining hierarchical reinforcement learning and Bayesian networks for natural language generation in situated dialogue. In Proceedings of the European Workshop on Natural Language Generation (ENLG), pages 110-120, Nancy, France. Association for Computational Linguistics.Google Scholar

Dethlefs, N. and Cuayáhuitl, H. (2011b). Hierarchical reinforcement learning and hidden Markov models for task-oriented natural language generation. In Proceedings of the Annual Meeting of the Association for Computational Linguistics: Human Language Technologies (ACL-HLT), pages 654-659, Portland, OR. Association for Computational Linguistics.Google Scholar

Dethlefs, N. and Cuayáhuitl, H. (2012). Comparing HMMs and Bayesian Networks for surface realisation. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), pages 636-640, Montréal, Canada. Association for Computational Linguistics.Google Scholar

Dethlefs, N., Cuayáhuitl, H., and Viethen, J. (2011). Optimising natural language generation decision making for situated dialogue. In Proceedings of the SIGdial Conference on Discourse and Dialogue (SIGDIAL), pages 78-87, Portland, OR. Association for Computational Linguistics.Google Scholar

DeVault, D., Traum, D., and Artstein, R. (2008). Practical grammar-based NLG from examples. In Proceedings of the International Workshop on Natural Language Generation (INLG), pages 77-85, Salt Fork, OH. Association for Computational Linguistics.Google Scholar

Dietterich, T. G. (2000). Hierarchical reinforcement learning with the MAXQ value function decomposition. Journal of Artificial Intelligence Research, 13:227-303.Google Scholar

Gargett, A., Garoufi, K., Koller, A., and Striegnitz, K. (2010). The GIVE-2 corpus of giving instructions in virtual environments. In Proceedings of the International Conference on Language Resources and Evaluation (LREC), Valletta, Malta. European Language Resources Association.Google Scholar

Garoufi, K. and Koller, A. (2010). Automated planning for situated natural language generation. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL), pages 1573-1582, Uppsala, Sweden. Association for Computational Linguistics.Google Scholar

Garoufl, K. and Roller, A. (2011). Combining symbolic and corpus-based approaches for the generation of successful referring expressions. In Proceedings of the European Workshop on Natural Language Generation (ENLG), pages 121-131. Nancy, France. Association for Computational Linguistics.Google Scholar

Gašić, M., Jurcicek, F., Thomson, B., Yu, K., and Young, S. (2011). On-line policy optimisation of spoken dialogue systems via live interaction with human subjects. In Proceedings of the IEEE workshop on Automatic Speech Recognition and Understanding (ASRU), pages 312-317, Waikoloa, HI. Institute of Electrical and Electronics Engineers.Google Scholar

Henderson, J., Lemon, O., and Georgila, K. (2005). Hybrid reinforcement/supervised learning for dialogue policies from Communicator data. In Proceedings of the Workshop on Knowledge and Reasoning in Practical Dialogue Systems (KRPDS), pages 68-75, Edinburgh, Scotland. International Joint Conference on Artificial Intelligence.Google Scholar

Janarthanam, S. and Lemon, O. (2010). Learning adaptive referring expression generation policies for spoken dialogue systems. In Krahmer, E. and Theune, M., editors, Empirical Methods in Natural Language Generation, pages 67-84. Springer, Berlin.Google Scholar

Janarthanam, S., Lemon, O., and Liu, X. (2012). A web-based evaluation framework for spatial instruction-giving systems. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL), pages 49-54, Jeju Island, Korea. Association for Computational Linguistics.Google Scholar

Kelleher, J. D., Kruijff, G.-J. M., and Costello, F. J. (2006). Incremental generation of spatial referring expressions in situated dialog. In Proceedings of the International Conference on Computational Linguistics and the Annual Meeting of the Association for Computational Linguistics (COLING-ACL), pages 745-752, Sydney, Australia. Association for Computational Linguistics.Google Scholar

Koller, A., Striegnitz, K., Byron, D., Cassell, J., Dale, R., and Moore, J. D. (2010a). The first challenge on generating instructions in virtual environments. In Krahmer, E. and Theune, M., editors, Empirical Methods in Natural Language Generation, pages 328-352. Springer LNCS, Berlin, Germany.Google Scholar

Koller, A., Striegnitz, K., Gargett, A., Byron, D., Cassell, J., Dale, R., Moore, J., and Oberlander, J. (2010b). Report on the second NLG challenge on generating instructions in virtual environments (GIVE-2). In Proceedings of the International Conference on Natural Language Generation (INLG), pages 243-250, Trim, Ireland. Association for Computational Linguistics.Google Scholar

Lemon, O. (2008). Adaptive natural language generation in dialogue using reinforcement learning. In Proceedings of the Workshop on the Semantics and Pragmatics of Dialogue (SEMDIAL), pages 141-148, London, UK. SemDial.Google Scholar

Lemon, O. (2011). Learning what to say and how to say it: Joint optimisation of spoken dialogue management and natural language generation. Computer Speech & Language, 25(2):210-221.CrossRef Google Scholar

Levin, E., Pieraccini, R., and Eckert, W. (2000). A stochastic model of human-machine interaction for learning dialog strategies. IEEE Transactions on Speech and Audio Processing, 8(1):11-23.CrossRef Google Scholar

Mairesse, F., Gašić, M., Jurčíček, F., Keizer, S., Thomson, B., Yu, K., and Young, S. (2010). Phrase-based statistical language generation using graphical models and active learning. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL), pages 1552-1561, Uppsala, Sweden. Association for Computational Linguistics.Google Scholar

Pietquin, O., Geist, M., Chandramohan, S., and Frezza-Buet, H. (2011). Sample-efficient batch reinforcement learning for dialogue management optimization. ACM Transactions on Speech and Language Processing (Special Issue on Machine Learning for Robust and Adaptive Spoken Dialogue Systems), 7(3):7.Google Scholar

Rabiner, L. R. (1989). Tutorial on hidden Markov models and selected applications in speech recognition. Proceedings of the IEEE, 77(2):257-286.CrossRef Google Scholar

Raux, A. and Nakano, M. (2010). The dynamics of action corrections in situated interaction. In Proceedings of the SIGdial Conference on Discourse and Dialogue (SIGDIAL), pages 165-174, Tokyo, Japan. Association for Computational Linguistics.Google Scholar

Reboul, A. (1998). A relevance theoretic approach to reference. In Proceedings of the Relevance Theory Workshop, pages 45-50, Luton, UK. University of Luton.Google Scholar

Rieser, V. and Lemon, O. (2011). Learning and evaluation of dialogue strategies for new applications: Empirical methods for optimization from small data sets. Computational Linguistics, 37(1):153-196.CrossRef Google Scholar

Salmon-Alt, S. and Romary, L. (2001). Reference resolution within the framework of cognitive grammar. In Proceedings of the International Colloquium on Cognitive Science, Donostia -San Sebastian, Spain. Institute for Logic, Cognition, Language and Information (ILCLI) and the Dept. of Logic and Philosophy of Science of the University of the Basque Country.Google Scholar

Scott, D. and Moore, J. (2007). An NLG evaluation competition? Eight reasons to be cautious. In Proceedings of the NSF Workshop onShared Tasks and Comparative EvaluationinNatural Language Generation, Arlington, VA. National Science Foundation.Google Scholar

Stent, A., Bangalore, S., and Di Fabbrizio, G. (2008). Where do the words come from? Learning models for word choice and ordering from spoken dialog corpora. In Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), pages 5037-5040, Las Vegas, NV. Institute of Electrical and Electronics Engineers.Google Scholar

Stoia, L., Shockley, D. M., Byron, D., and Fosler-Lussier, E. (2006). Noun phrase generation for situated dialogs. In Proceedings of the International Conference on Natural Language Generation (INLG), pages 81-88, Sydney, Australia. Association for Computational Linguistics.Google Scholar

Sutton, R. (1996). Generalization in reinforcement learning: Successful examples using sparse coarse coding. In Touretzky, D., Mozer, M., and Hasselmo, M., editors, Advances in Neural Information Processing Systems, pages 1038-1044. MIT Press, Cambridge, MA.Google Scholar

Sutton, R. S. and Barto, A. G. (1998). Reinforcement Learning: An Introduction. MIT Press, Cambridge, MA.Google Scholar

Thomson, B. (2009). Statistical Methods for Spoken Dialogue Management. PhD thesis, University of Cambridge.Google Scholar

van Zaanen, M. (2000). Bootstrapping syntax and recursion using alignment-based learning. In Proceedings of the International Conference on Machine Learning (ICML), pages 1063-1070, Stanford, CA. The International Machine Learning Society.Google Scholar

Watkins, C. (1989). Learning from Delayed Rewards. PhD thesis, Kings College, University of Cambridge.Google Scholar

Williams, J. D. (2006). Partially Observable Markov Decision Processes for Spoken Dialogue Management. PhD thesis, University of Cambridge.Google Scholar

Young, S., Gašić, M., Keizer, S., Mairesse, F., Schatzmann, J., Thomson, B., and Yu, K. (2010). The hidden information state model: A practical framework for POMDP-based spoken dialogue management. Computer Speech & Language, 24(2):150-174.CrossRef Google Scholar

Accessibility standard: Unknown

Accessibility compliance for the PDF of this book is currently unknown and may be updated in the future.