Cardiff University | Prifysgol Caerdydd ORCA
Online Research @ Cardiff 
WelshClear Cookie - decide language by browser settings

SHERLOCK: Experimental evaluation of a conversational agent for mobile information tasks

Preece, Alun, Webberley, William, Braines, David, Zaroukian, Erin G. and Bakdash, Jonathan Z. 2017. SHERLOCK: Experimental evaluation of a conversational agent for mobile information tasks. IEEE Transactions on Human-Machine Systems 47 (6) , pp. 1017-1028. 10.1109/THMS.2017.2700625

[img]
Preview
PDF - Published Version
Available under License Creative Commons Attribution.

Download (573kB) | Preview

Abstract

Abstract—Controlled Natural Language (CNL) has great potential to support human-machine interaction (HMI) because it provides an information representation that is both human readable and machine processable.We investigated the effectiveness of a CNL-based conversational interface for HMI in a behavioural experiment called Simple Human Experiment Regarding Locally Observed Collective Knowledge (SHERLOCK). In SHERLOCK, individuals acted in groups to discover and report information to the machine using natural language (NL), which the machine then processed into CNL. The machine fused responses from different users to form a common operating picture, a dashboard showing the level of agreement for distinct information. To obtain information to add to this dashboard, users explored the real world in a simulated crowd-sourced sensing scenario. This scenario represented a simplified, controlled analogue for tactical intelligence (i.e., direct intelligence of the environment), which is key for rapidly planning military, law enforcement, and emergency operations. Overall, despite close to zero training, 74% of the users inputted NL that was machine interpretable and addressed the assigned tasks. An experimental manipulation aimed to increase user-machine interaction, however, did not improve performance as hypothesised. Nevertheless, results indicate the conversational interface may be effective in assisting humans with collection and fusion of information in a crowd-sourcing context.

Item Type: Article
Date Type: Publication
Status: Published
Schools: Computer Science & Informatics
Crime and Security Research Institute (CSURI)
Uncontrolled Keywords: Controlled natural language (CNL), conversational interface, human–computer collaboration (HCC), human–machine interaction (HMI), tactical intelligence Collaboration, Usability, Sensors, Training, Speech, Natural languages, Context
Publisher: Institute of Electrical and Electronics Engineers (IEEE)
ISSN: 2168-2305
Funders: U.S. and U.K. Governments
Last Modified: 16 Nov 2017 09:34
URI: http://orca.cf.ac.uk/id/eprint/100131

Actions (repository staff only)

Edit Item Edit Item

Full Text Downloads from ORCA for this publication

Top Downloads of this item by Country

Monthly Full Text Downloads of this item

More statistics for this item...