Natural language generation for augmentative and assistive technologies

doi:10.1017/CBO9780511844492.011

11 - Natural language generation for augmentative and assistive technologies

from Part IV - Engagement

Published online by Cambridge University Press: 05 July 2014

Nava Tintarev ,

Ehud Reiter ,

Rolf Black and

Annalu Waller

Edited by

Amanda Stent and

Srinivas Bangalore

Show author details

Nava Tintarev: Affiliation:
University of Aberdeen
Ehud Reiter: Affiliation:
University of Aberdeen
Rolf Black: Affiliation:
Dundee University
Annalu Waller: Affiliation:
University of Dundee
Amanda Stent: Affiliation:
AT&T Research, Florham Park, New Jersey
Srinivas Bangalore: Affiliation:
AT&T Research, Florham Park, New Jersey

Book contents

Get access

Summary

Introduction

Many people are not able to communicate easily, because of diversity in their physical and/or intellectual abilities. For example, someone with cerebral palsy may not have enough control over their vocal tract and mouth to speak (or enough manual dexterity control to use sign language); and someone with autism may be physically capable, but unable to speak because of their cognitive profile.

Augmentative and alternative communication (AAC) is a sub-field of assistive technology focusing on tools to help people with major motor and/or cognitive impairments communicate better. Traditionally, AAC has used technology to provide speech output for non-speaking users, either by converting text to speech or speaking out pre-stored phrases. The initial focus of computer-based AAC systems was on giving users access to a broad set of symbols/pictures (by using some kind of menu or search system), and on using speech synthesis to automatically speak out phrases corresponding to the selected symbols. However, computers can in principle do much more than this.

In particular, since many AAC systems are intended to help users produce utterances, there seems to be a natural role for Natural Language Generation (NLG) to play, as it is after all a technology for producing language. However, the use of NLG in AAC is somewhat different from most uses of NLG. Since the goal of AAC is to help the user communicate, the NLG system must be used interactively, under the user’s control; we want to assist the user in communication, not replace the user with an automatic communicator. Also, since most human communication is social, NLG AAC systems often need to generate texts whose communicative goal is social interaction. Hence, NLG AAC systems are very different from systems that summarize information in a task-oriented context, which has been the focus of most NLG research for interactive systems.

Information

Type: Chapter
Information: Natural Language Generation in Interactive Systems , pp. 252 - 278

DOI: https://doi.org/10.1017/CBO9780511844492.011 [Opens in a new window]

Publisher: Cambridge University Press

Print publication year: 2014

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Book purchase

Temporarily unavailable

References

American Speech-Language-Hearing Association (2002). Augmentative and alternative communication: Knowledge and skills for service delivery [knowledge and skills]. Available from http://www.asha.org/policy. Accessed on 11/24/2013.

Andreasen, P., Waller, A., and Gregor, P. (1998). BlissWord – full access to Blissymbols for all users. In Proceedings of the Biennial Conference of the International Society for Augmentative and Alternative Communication, pages 167-168, Dublin, Ireland. International Society for Augmentative and Alternative Communication.Google Scholar

Beukelman, D. R. and Mirenda, P. (1998). Augmentative and Alternative Communication: Management of Severe Communication Disorders in Children and Adults. Brookes Publishing Company, Baltimore, MD, 2nd edition.Google Scholar

Biswas, P. (2006). A flexible approach to natural language generation for disabled children. In Proceedings of the International Conference on Computational Linguistics and the Annual Meeting of the Association for Computational Linguistics (COLING-ACL), pages 1-6, Sydney, Australia. Association for Computational Linguistics.Google Scholar

Biswas, P. and Samanta, D. (2008). Friend: A communication aid for persons with disabilities. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 16(2):205-209.CrossRef Google Scholar PubMed

Black, R., Reddington, J., Reiter, E., Tintarev, N., and Waller, A. (2010). Using NLG and sensors to support personal narrative for children with complex communication needs. In Proceedings of the NAACL HLT Workshop on Speech and Language Processing for Assistive Technologies, pages 1-9, Los Angeles, CA. Association for Computational Linguistics.Google Scholar

de Rosis, F. and Grasso, F. (2000). Affective natural language generation. In Paiva, A., editor, Affective Interactions, pages 204-218. Springer LNCS, Berlin, Germany.Google Scholar

Demasco, P. W. and McCoy, K. F. (1992). Generating text from compressed input: An intelligent interface for people with severe motor impairments. Communications of the ACM, 35(5): 68-78.CrossRef Google Scholar

Demir, S., Carberry, S., and McCoy, K. F. (2011). Summarizing information graphics textually. Computational Linguistics, 38(3):527-574.Google Scholar

Dempster, M. (2008). Using natural language generation to encourage effective communication in non-speaking people. In Proceedings of the Young Researchers Consortium at the International Conference on Computers Helping People with Special Needs, Linz, Austria.Google Scholar

Dempster, M., Alm, N., and Reiter, E. (2010). Automatic generation of conversational utterances and narrative for augmentative and alternative communication: A prototype system. In Proceedings of the NAACL HLT Workshop on Speech and Language Processing for Assistive Technologies, pages 10-18, Los Angeles, CA. Association for Computational Linguistics.Google Scholar

Dugard, P., File, P., and Todman, J. (2011). Single-case and Small-n Experimental Designs: A Practical Guide To Randomization Tests. Routledge, New York, NY, USA, 2nd edition.Google Scholar

Ferres, L., Lindgaard, G., and Sumegi, L. (2013). Evaluating a tool for improving accessibility to charts and graphs. ACM Transactions on Computer-Human Interaction (TOCHI), 20(5).CrossRef Google Scholar

Gatt, A. and Reiter, E. (2009). SimpleNLG: A realisation engine for practical applications. In Proceedings of the European Workshop on Natural Language Generation (ENLG), pages 90-93, Athens, Greece. Association for Computational Linguistics.Google Scholar

Grove, N. (2010). The Big Book of Storysharing. Special Educational Needs Joint Initiative for Training (SENJIT), Institute of Education, University of London, London, UK.Google Scholar

Light, J. and Drager, K. (2007). AAC technologies for young children with complex communication needs: State of the science and future research directions. Augmentative and Alternative Communication, 23(3):204-216.Google Scholar PubMed

Liu, X., Sen, A., Bauer, J., and Zitzmann, C. (2007). A software client for wi-fi based realtime location tracking of patients. In Proceedings of the International Conference on Medical Imaging and Informatics, pages 141-150, Beijing, China. Springer.Google Scholar

McCabe, A. and Peterson, C. (1991). Getting the story: A longitudinal study of parental styles in eliciting narratives and developing narrative skill. In McCabe, A. and Peterson, C., editors, Developing Narrative Structure, pages 217-253. Lawrence Erlbaum Associates, Hillsdale, NJ.Google Scholar

McCoy, K. F., Bedrosian, J., and Hoag, L. (2010). Implications of pragmatic and cognitive theories on the design of utterance-based AAC systems. In Proceedings of the NAACL HLT Workshop on Speech and Language Processing for Assistive Technologies, pages 19-27, Los Angeles, CA. Association for Computational Linguistics.Google Scholar

Mellish, C. and Dale, R. (1998). Evaluation in the context of natural language generation. Computer Speech & Language, 12(4):349-373.CrossRef Google Scholar

Newell, A. F., Carmichael, A., Gregor, P., Alm, N., and Waller, A. (2008). Information technology for cognitive support. In Sears, A. and Jacko, J. A., editors, The Human-Computer Interaction Handbook: Fundamentals, Evolving Technologies and Emerging Applications, pages 811-828. Lawrence Erlbaum Associates, Mahwah, NJ, 2nd edition.Google Scholar

Nikolova, S., Tremaine, M., and Cook, P. (2010). Click on bake to get cookies: Guiding word-finding with semantic associations. In Proceedings of the ACM SIGACCESS Conference on Computers and Accessibility (ASSETS), pages 155-162, Orlando, FL. Association for Computing Machinery.Google Scholar

Polkinghorne, D. (1995). Narrative configuration in qualitative analysis. In Hatch, J. A. and Wisniewski, R., editors, Life History and Narrative, pages 5-24. Taylor & Francis, Bristol, PA.Google Scholar

Portet, F., Reiter, E., Gatt, A., Hunter, J., Sripada, S., Freer, Y., and Sykes, C. (2009). Automatic generation of textual summaries from neonatal intensive care data. Artificial Intelligence, 173(7-8):789-816.CrossRef Google Scholar

Quasthoff, U. M. and Nikolaus, K. (1982). What makes a good story? Towards the production of conversational narratives. In Flammer, A. and Kintsch, W., editors, Discourse Processing. North-Holland Publishing Company, Amsterdam.Google Scholar

Reddington, J. and Tintarev, N. (2011). Automatically generating stories from sensor data. In Proceedings of the International Conference on Intelligent User Interfaces (IUI), pages 407-410, Palo Alto, CA. Association for Computing Machinery.Google Scholar

Reichenbach, H. (1947). Elements of Symbolic Logic. Macmillan Co., New York, NY.Google Scholar

Reiter, E. (2007). An architecture for data-to-text systems. In Proceedings of the European Workshop on Natural Language Generation (ENLG), pages 97-104, Saarbrücken, Germany. Association for Computational Linguistics.Google Scholar

Reiter, E. and Belz, A. (2009). An investigation into the validity of some metrics for automatically evaluating NLG systems. Computational Linguistics, 35(4):529-558.CrossRef Google Scholar

Reiter, E., Gatt, A., Portet, F., and van der Meulen, M. (2008). The importance of narrative and other lessons from an evaluation of an NLG system that summarises clinical data. In Proceedings of the International Conference on Natural Language Generation (INLG), pages 147-156, Salt Fork, OH. Association for Computational Linguistics.Google Scholar

Roark, B., de Villiers, J., Gibbons, C., and Fried-Oken, M. (2010). Scanning methods and language modeling for binary switch typing. In Proceedings of the NAACL HLT Workshop on Speech and Language Processing for Assistive Technologies, pages 28-36, Los Angeles, CA. Association for Computational Linguistics.Google Scholar

Robertson, J. and Good, J. (2005). Story creation in virtual game worlds. Commununications of the ACM, 48(1):61-65.CrossRef Google Scholar

Thomas, K. E. and Sripada, S. (2008). What's in a message? Interpreting geo-referenced data for the visually-impaired. In Proceedings of the International Conference on Natural Language Generation (INLG), pages 113-120, Salt Fork, OH. Association for Computational Linguistics.Google Scholar

Todman, J. and Alm, N. (2003). Modelling conversational pragmatics in communication aids. Journal of Pragmatics, 35(4):523-538.CrossRef Google Scholar

Todman, J., Alm, N., Higginbotham, J., and File, P. (2008). Whole utterance approaches in AAC. Augmentative and Alternative Communication, 24(3):235-254.CrossRef Google Scholar PubMed

Waller, A. (2006). Personal narrative and AAC: From transactional to interactional conversation. In Proceedings of the International Society for Augmentative and Alternative Communication (ISAAC) Research Symposium, Dtlsseldorf, Germany. International Society for Augmentative and Alternative Communication.Google Scholar

Waller, A., Dennis, F., Brodie, J., and Cairns, A. Y. (1998a). Evaluating the use of TalksBac, a predictive communication device for nonfluent adults with aphasia. International Journal of Language & Communication Disorders, 33(1):45-70.Google Scholar PubMed

Waller, A., Dennis, F., Cairns, A., Whitehead, N., Brodie, J., Newell, A. F., and Morrison, K. (1998b). The future development of TalksBac: A predictive augmentative communication system for adults with nonfluent aphasia. International Journal of Language & Communication Disorders, 33(1):45-70.Google Scholar

Waller, A., Francis, J., Tait, L., Booth, L., and Hood, H. (1999). The WriteTalk project: Story-based interactive communication. In Buhler, C. and Knops, H., editors, Assistive Technology on the Threshold of the New Millennium, pages 180-184. IOS Press, Amsterdam, The Netherlands.Google Scholar

Waller, A. and Newell, A. (1997). Towards a narrative-based communication systems. European Journal of Disorders of Communication, 32(S3):289-306.CrossRef Google Scholar

Waller, A., O'Mara, D., Manurung, R., Pain, H., and Ritchie, G. (2005). Facilitating user feedback in the design of a novel joke generation system for people with severe communication impairment. In Proceedings of the International Conference on Human-Computer Interaction (HCII), Las Vegas, NV. Lawrence Erlbaum Associates.Google Scholar

Waller, A., O'Mara, D. A., Tait, L., Booth, L., Brophy-Arnott, B., and Hood, H. E. (2001). Using written stories to support the use of narrative in conversational interactions: Case study. Augmentative and Alternative Communication, 17(4):221-232.CrossRef Google Scholar

Yarrington, D., Pennington, C., Bunnell, H. T., Gray, J., Lilley, J., Nagao, K., and Polikoff, J. (2008). ModelTalker Voice Recorder (MTVR) - a system for capturing individual voices for synthetic speech. In Proceedings of the ISAAC Biennial Conference, Montreal, Canada. International Society for Augmentative and Alternative Communication.Google Scholar

Accessibility standard: Unknown

Accessibility compliance for the PDF of this book is currently unknown and may be updated in the future.