Improving distributed representation of word sense via WordNet Gloss composition and context clustering

Tao Chen, Ruifeng Xu, Yulan He, Xuan Wang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

In recent years, there has been an increas-ing interest in learning a distributed rep-resentation of word sense. Traditional context clustering based models usually require careful tuning of model parame-ters, and typically perform worse on infre-quent word senses. This paper presents a novel approach which addresses these lim-itations by first initializing the word sense embeddings through learning sentence-level embeddings from WordNet glosses using a convolutional neural networks. The initialized word sense embeddings are used by a context clustering based model to generate the distributed representations of word senses. Our learned represen-tations outperform the publicly available embeddings on 2 out of 4 metrics in the word similarity task, and 6 out of 13 sub tasks in the analogical reasoning task.

Original languageEnglish
Title of host publicationProceedings of the 53rd annual meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Short Papers)
PublisherAssociation for Computational Linguistics
Pages15-20
Number of pages6
Volume2
ISBN (Print)978-1-941643-73-0
Publication statusPublished - 2015
Event53rd annual meeting of the Association for Computational Linguistics / 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing - Beijing, China
Duration: 26 Jul 201531 Jul 2015

Meeting

Meeting53rd annual meeting of the Association for Computational Linguistics / 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing
Abbreviated titleACL-IJCNLP 2015
CountryChina
CityBeijing
Period26/07/1531/07/15

Fingerprint

Chemical analysis
Tuning
Neural networks

Cite this

Chen, T., Xu, R., He, Y., & Wang, X. (2015). Improving distributed representation of word sense via WordNet Gloss composition and context clustering. In Proceedings of the 53rd annual meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Short Papers) (Vol. 2, pp. 15-20). Association for Computational Linguistics.
Chen, Tao ; Xu, Ruifeng ; He, Yulan ; Wang, Xuan. / Improving distributed representation of word sense via WordNet Gloss composition and context clustering. Proceedings of the 53rd annual meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Short Papers). Vol. 2 Association for Computational Linguistics, 2015. pp. 15-20
@inproceedings{a42575ca082741f2b046e7a1f11b453a,
title = "Improving distributed representation of word sense via WordNet Gloss composition and context clustering",
abstract = "In recent years, there has been an increas-ing interest in learning a distributed rep-resentation of word sense. Traditional context clustering based models usually require careful tuning of model parame-ters, and typically perform worse on infre-quent word senses. This paper presents a novel approach which addresses these lim-itations by first initializing the word sense embeddings through learning sentence-level embeddings from WordNet glosses using a convolutional neural networks. The initialized word sense embeddings are used by a context clustering based model to generate the distributed representations of word senses. Our learned represen-tations outperform the publicly available embeddings on 2 out of 4 metrics in the word similarity task, and 6 out of 13 sub tasks in the analogical reasoning task.",
author = "Tao Chen and Ruifeng Xu and Yulan He and Xuan Wang",
year = "2015",
language = "English",
isbn = "978-1-941643-73-0",
volume = "2",
pages = "15--20",
booktitle = "Proceedings of the 53rd annual meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Short Papers)",
publisher = "Association for Computational Linguistics",

}

Chen, T, Xu, R, He, Y & Wang, X 2015, Improving distributed representation of word sense via WordNet Gloss composition and context clustering. in Proceedings of the 53rd annual meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Short Papers). vol. 2, Association for Computational Linguistics, pp. 15-20, 53rd annual meeting of the Association for Computational Linguistics / 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing, Beijing, China, 26/07/15.

Improving distributed representation of word sense via WordNet Gloss composition and context clustering. / Chen, Tao; Xu, Ruifeng; He, Yulan; Wang, Xuan.

Proceedings of the 53rd annual meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Short Papers). Vol. 2 Association for Computational Linguistics, 2015. p. 15-20.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Improving distributed representation of word sense via WordNet Gloss composition and context clustering

AU - Chen, Tao

AU - Xu, Ruifeng

AU - He, Yulan

AU - Wang, Xuan

PY - 2015

Y1 - 2015

N2 - In recent years, there has been an increas-ing interest in learning a distributed rep-resentation of word sense. Traditional context clustering based models usually require careful tuning of model parame-ters, and typically perform worse on infre-quent word senses. This paper presents a novel approach which addresses these lim-itations by first initializing the word sense embeddings through learning sentence-level embeddings from WordNet glosses using a convolutional neural networks. The initialized word sense embeddings are used by a context clustering based model to generate the distributed representations of word senses. Our learned represen-tations outperform the publicly available embeddings on 2 out of 4 metrics in the word similarity task, and 6 out of 13 sub tasks in the analogical reasoning task.

AB - In recent years, there has been an increas-ing interest in learning a distributed rep-resentation of word sense. Traditional context clustering based models usually require careful tuning of model parame-ters, and typically perform worse on infre-quent word senses. This paper presents a novel approach which addresses these lim-itations by first initializing the word sense embeddings through learning sentence-level embeddings from WordNet glosses using a convolutional neural networks. The initialized word sense embeddings are used by a context clustering based model to generate the distributed representations of word senses. Our learned represen-tations outperform the publicly available embeddings on 2 out of 4 metrics in the word similarity task, and 6 out of 13 sub tasks in the analogical reasoning task.

UR - http://www.scopus.com/inward/record.url?scp=84944073513&partnerID=8YFLogxK

UR - http://www.aclweb.org/anthology/P15-2003

M3 - Conference contribution

AN - SCOPUS:84944073513

SN - 978-1-941643-73-0

VL - 2

SP - 15

EP - 20

BT - Proceedings of the 53rd annual meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Short Papers)

PB - Association for Computational Linguistics

ER -

Chen T, Xu R, He Y, Wang X. Improving distributed representation of word sense via WordNet Gloss composition and context clustering. In Proceedings of the 53rd annual meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Short Papers). Vol. 2. Association for Computational Linguistics. 2015. p. 15-20