Learning task specific distributed paragraph representations using a 2-tier convolutional neural network

Tao Chen, Ruifeng Xu*, Yulan He, Xuan Wang

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference publication

Abstract

We introduce a type of 2-tier convolutional neural network model for learning distributed paragraph representations for a special task (e.g. paragraph or short document level sentiment analysis and text topic categorization). We decompose the paragraph semantics into 3 cascaded constitutes: word representation, sentence composition and document composition. Specifically, we learn distributed word representations by a continuous bag-of-words model from a large unstructured text corpus. Then, using these word representations as pre-trained vectors, distributed task specific sentence representations are learned from a sentence level corpus with task-specific labels by the first tier of our model. Using these sentence representations as distributed paragraph representation vectors, distributed paragraph representations are learned from a paragraph-level corpus by the second tier of our model. It is evaluated on DBpedia ontology classification dataset and Amazon review dataset. Empirical results show the effectiveness of our proposed learning model for generating distributed paragraph representations.

Original languageEnglish
Title of host publicationNeural information processing
Subtitle of host publication22nd International Conference, ICONIP 2015, Istanbul, Turkey, November 9-12, 2015, Proceedings, Part I
Place of PublicationCham (CH)
PublisherSpringer
Pages467-475
Number of pages9
ISBN (Electronic)978-3-319-26532-2
ISBN (Print)978-3-319-26531-5
DOIs
Publication statusPublished - 12 Nov 2015
Event22nd International Conference on Neural Information Processing - Istanbul, Turkey
Duration: 9 Nov 201512 Nov 2015

Publication series

NameLecture notes in computer science
PublisherSpringer
Volume9489
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference22nd International Conference on Neural Information Processing
Abbreviated titleICONIP 2015
CountryTurkey
CityIstanbul
Period9/11/1512/11/15

Keywords

  • convolutional neural network
  • distributed representation
  • natural language processing

Fingerprint Dive into the research topics of 'Learning task specific distributed paragraph representations using a 2-tier convolutional neural network'. Together they form a unique fingerprint.

Cite this