SpaceGTN: A time-agnostic graph transformer network for handwritten diagram recognition and segmentation

Hu, Haoxiang, Gao, Cangjun, Li, Yaokun, Deng, Xiaoming, Lai, YuKun ORCID: https://orcid.org/0000-0002-2094-5680, Ma, Cuixia, Liu, Yong-Jin and Wang, Hongan 2024. SpaceGTN: A time-agnostic graph transformer network for handwritten diagram recognition and segmentation. Presented at: The Thirty-Eighth AAAI Conference on Artificial Intelligence (AAAI-24), 20-27 February 2024. Proceedings of the AAAI Conference on Artificial Intelligence. , vol.38 (3) Association for the Advancement of Artificial Intelligence, pp. 2211-2219. 10.1609/aaai.v38i3.27994 Item availability restricted.

PDF - Accepted Post-Print Version Restricted to Repository staff only until 12 May 2024 due to copyright restrictions. Download (7MB)

Abstract

Online handwriting recognition is pivotal in domains like note-taking, education, healthcare, and office tasks. Existing diagram recognition algorithms mainly rely on the temporal information of strokes, resulting in a decline in recognition performance when dealing with notes that have been modified or have no temporal information. The current datasets are drawn based on templates and cannot reflect the real free-drawing situation. To address these challenges, we present SpaceGTN, a time-agnostic Graph Transformer Network, leveraging spatial integration and removing the need for temporal data. Extensive experiments on multiple datasets have demonstrated that our method consistently outperforms existing methods and achieves state-of-the-art performance. We also propose a pipeline that seamlessly connects offline and online handwritten diagrams. By integrating a stroke restoration technique with SpaceGTN, it enables intelligent editing of previously uneditable offline diagrams at the stroke level. In addition, we have also launched the first online handwritten diagram dataset, OHSD, which is collected using a free-drawing method and comes with modification annotations.

Item Type:	Conference or Workshop Item (Paper)
Date Type:	Published Online
Status:	Published
Schools:	Computer Science & Informatics
Publisher:	Association for the Advancement of Artificial Intelligence
ISSN:	2374-3468
Date of First Compliant Deposit:	2 April 2024
Date of Acceptance:	9 December 2023
Last Modified:	12 Apr 2024 15:30
URI:	https://orca.cardiff.ac.uk/id/eprint/167658

Actions (repository staff only)

Edit Item