SpeechMirror: A multimodal visual analytics system for personalized reflection of online public speaking effectiveness

Huang, Zeyuan, He, Qiang, Maher, Kevin, Deng, Xiaoming, Lai, Yukun ORCID: https://orcid.org/0000-0002-2094-5680, Ma, Cuixia, Qin, Sheng-feng, Liu, Yong-Jin and Wang, Hongan 2024. SpeechMirror: A multimodal visual analytics system for personalized reflection of online public speaking effectiveness. IEEE Transactions on Visualization and Computer Graphics 30 (1) , pp. 606-616. 10.1109/TVCG.2023.3326932

Preview

PDF - Accepted Post-Print Version Download (18MB) | Preview

Abstract

As communications are increasingly taking place virtually, the ability to present well online is becoming an indispensable skill. Online speakers are facing unique challenges in engaging with remote audiences. However, there has been a lack of evidence-based analytical systems for people to comprehensively evaluate online speeches and further discover possibilities for improvement. This paper introduces SpeechMirror, a visual analytics system facilitating reflection on a speech based on insights from a collection of online speeches. The system estimates the impact of different speech techniques on effectiveness and applies them to a speech to give users awareness of the performance of speech techniques. A similarity recommendation approach based on speech factors or script content supports guided exploration to expand knowledge of presentation evidence and accelerate the discovery of speech delivery possibilities. SpeechMirror provides intuitive visualizations and interactions for users to understand speech factors. Among them, SpeechTwin, a novel multimodal visual summary of speech, supports rapid understanding of critical speech factors and comparison of different speech samples, and SpeechPlayer augments the speech video by integrating visualization of the speaker's body language with interaction, for focused analysis. The system utilizes visualizations suited to the distinct nature of different speech factors for user comprehension. The proposed system and visualization techniques were evaluated with domain experts and amateurs, demonstrating usability for users with low visualization literacy and its efficacy in assisting users to develop insights for potential improvement.

Item Type:	Article
Date Type:	Publication
Status:	Published
Schools:	Computer Science & Informatics
Publisher:	Institute of Electrical and Electronics Engineers
ISSN:	1077-2626
Date of First Compliant Deposit:	25 September 2023
Date of Acceptance:	16 July 2023
Last Modified:	19 Feb 2024 16:27
URI:	https://orca.cardiff.ac.uk/id/eprint/162712

Actions (repository staff only)

Edit Item