Cardiff University | Prifysgol Caerdydd ORCA
Online Research @ Cardiff 
WelshClear Cookie - decide language by browser settings

Word embedding as maximum a posteriori estimation

Jameel, Shoaib, Fu, Zihao, Shi, Bei, Lam, Wai and Schockaert, Steven 2019. Word embedding as maximum a posteriori estimation. Presented at: AAAI-19: 33rd AAAI Conference on Artificial Intelligence, Honolulu, HI, USA, 27 January - 1 February 2019. Proceedings of the AAAI Conference on Artificial Intelligence. , vol. 1. Palo Alto, California: AAAI Press, pp. 6562-6569. 10.1609/aaai.v33i01.33016562

[img]
Preview
PDF - Accepted Post-Print Version
Download (264kB) | Preview

Abstract

The GloVe word embedding model relies on solving a global optimization problem, which can be reformulated as a maximum likelihood estimation problem. In this paper, we propose to generalize this approach to word embedding by considering parametrized variants of the GloVe model and incorporating priors on these parameters. To demonstrate the usefulness of this approach, we consider a word embedding model in which each context word is associated with a corresponding variance, intuitively encoding how informative it is. Using our framework, we can then learn these variances together with the resulting word vectors in a unified way. We experimentally show that the resulting word embedding models outperform GloVe, as well as many popular alternatives.

Item Type: Conference or Workshop Item (Paper)
Date Type: Publication
Status: Published
Schools: Computer Science & Informatics
Publisher: AAAI Press
ISBN: 978-1-57735-809-1
ISSN: 2374-3468
Date of First Compliant Deposit: 6 February 2019
Last Modified: 28 Aug 2019 09:12
URI: http://orca.cf.ac.uk/id/eprint/117380

Actions (repository staff only)

Edit Item Edit Item

Downloads

Downloads per month over past year

View more statistics