2 results
Actionable conversational quality indicators for improving task-oriented dialog systems
- Michael Higgins, Dominic Widdows, Beth Ann Hockey, Akshay Hazare, Kristen Howell, Gwen Christian, Sujit Mathi, Chris Brew, Andrew Maurer, George Bonev, Matthew Dunn, Joseph Bradley
-
- Journal:
- Natural Language Engineering , First View
- Published online by Cambridge University Press:
- 09 January 2024, pp. 1-26
-
- Article
-
- You have access Access
- Open access
- HTML
- Export citation
-
Automatic dialog systems have become a mainstream part of online customer service. Many such systems are built, maintained, and improved by customer service specialists, rather than dialog systems engineers and computer programmers. As conversations between people and machines become commonplace, it is critical to understand what is working, what is not, and what actions can be taken to reduce the frequency of inappropriate system responses. These analyses and recommendations need to be presented in terms that directly reflect the user experience rather than the internal dialog processing.
This paper introduces and explains the use of Actionable Conversational Quality Indicators (ACQIs), which are used both to recognize parts of dialogs that can be improved and to recommend how to improve them. This combines benefits of previous approaches, some of which have focused on producing dialog quality scoring while others have sought to categorize the types of errors the dialog system is making. We demonstrate the effectiveness of using ACQIs on LivePerson internal dialog systems used in commercial customer service applications and on the publicly available LEGOv2 conversational dataset. We report on the annotation and analysis of conversational datasets showing which ACQIs are important to fix in various situations.
The annotated datasets are then used to build a predictive model which uses a turn-based vector embedding of the message texts and achieves a 79% weighted average f1-measure at the task of finding the correct ACQI for a given conversation. We predict that if such a model worked perfectly, the range of potential improvement actions a bot-builder must consider at each turn could be reduced by an average of 81%.
three - Negotiating unfamiliar environments
- Edited by Alan Walker, The University of Sheffield
-
- Book:
- The New Dynamics of Ageing Volume 2
- Published by:
- Bristol University Press
- Published online:
- 13 April 2022
- Print publication:
- 25 July 2018, pp 35-50
-
- Chapter
- Export citation
-
Summary
Introduction
‘Active and healthy ageing’, with the goal of staying as independent as possible for as long as possible, has continued to be the policy focus in many countries (WHO, 2015). ‘Ageing in place’ and individual responsibility have also become enshrined in policy, reinforcing the importance of designing age-friendly communities that support independent living. To feel safe and comfortable in one's local neighbourhood with access to a variety of activities are crucial factors in retaining independence in later life. ‘Ageing in place’ and ‘place attachment’, another concept well used in ageing research (Smith, 2009), however, assumes some familiarity over time with that place.
Age-friendly cities are viewed as sensitive to age from the perspective of older residents living there. Although this is a critical component, which should be at the heart of all considerations of town planning, increasingly there is a need to assess the environment from the perspective of a visitor or someone who is unfamiliar with the environment. There are three major reasons that older people are increasingly experiencing environments that can be unfamiliar to them. This may be because of travelling as tourists to new areas; urban regeneration; or as a result of cognitive decline, where the familiar becomes unfamiliar.
The central aim of the project was to determine the mechanisms and strategies used by older people to navigate unfamiliar spaces as pedestrians (‘unfamiliar’ defined as new spaces to the older person or spaces that have become unfamiliar). Although there are many studies on accessibility (Granborn et al, 2016; I’DGO, no date), there is less research on the impact and effects of architecture and town design on older people's walkability, usability or their perception of the unfamiliar built environment.
The effects of the built environment and use of space on older people's self-perception and identity are being increasingly recognised (Peace et al, 2006). As people go through the life course, their use of space changes (Rowles, 1978).
![](/core/cambridge-core/public/images/lazy-loader.gif)