A convolutional attention model for text classification

Jiachen Du, Lin Gui, Ruifeng Xu, Yulan He

Research output: Chapter in Book/Report/Conference proceedingOther chapter contribution

Abstract

Neural network models with attention mechanism have shown their efficiencies on various tasks. However, there is little research work on attention mechanism for text classification and existing attention model for text classification lacks of cognitive intuition and mathematical explanation. In this paper, we propose a new architecture of neural network based on the attention model for text classification. In particular, we show that the convolutional neural network (CNN) is a reasonable model for extracting attentions from text sequences in mathematics. We then propose a novel attention model base on CNN and introduce a new network architecture which combines recurrent neural network with our CNN-based attention model. Experimental results on five datasets show that our proposed models can accurately capture the salient parts of sentences to improve the performance of text classification.
Original languageEnglish
Title of host publicationEnglish
Pages183-195
DOIs
Publication statusPublished - 5 Jan 2018
Event6th CCF International Conference - Dalian, China
Duration: 8 Nov 201712 Nov 2017

Publication series

NameNatural Language Processing and Chinese Computing
Volume10619
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference6th CCF International Conference
CountryChina
CityDalian
Period8/11/1712/11/17

Fingerprint

Neural networks
Recurrent neural networks
Network architecture

Cite this

Du, J., Gui, L., Xu, R., & He, Y. (2018). A convolutional attention model for text classification. In English (pp. 183-195). [Chapter 16] (Natural Language Processing and Chinese Computing; Vol. 10619). https://doi.org/10.1007/978-3-319-73618-1_16
Du, Jiachen ; Gui, Lin ; Xu, Ruifeng ; He, Yulan. / A convolutional attention model for text classification. English. 2018. pp. 183-195 (Natural Language Processing and Chinese Computing).
@inbook{4a54e3e66a78430892072769c5c7476a,
title = "A convolutional attention model for text classification",
abstract = "Neural network models with attention mechanism have shown their efficiencies on various tasks. However, there is little research work on attention mechanism for text classification and existing attention model for text classification lacks of cognitive intuition and mathematical explanation. In this paper, we propose a new architecture of neural network based on the attention model for text classification. In particular, we show that the convolutional neural network (CNN) is a reasonable model for extracting attentions from text sequences in mathematics. We then propose a novel attention model base on CNN and introduce a new network architecture which combines recurrent neural network with our CNN-based attention model. Experimental results on five datasets show that our proposed models can accurately capture the salient parts of sentences to improve the performance of text classification.",
author = "Jiachen Du and Lin Gui and Ruifeng Xu and Yulan He",
year = "2018",
month = "1",
day = "5",
doi = "10.1007/978-3-319-73618-1_16",
language = "English",
series = "Natural Language Processing and Chinese Computing",
pages = "183--195",
booktitle = "English",

}

Du, J, Gui, L, Xu, R & He, Y 2018, A convolutional attention model for text classification. in English., Chapter 16, Natural Language Processing and Chinese Computing, vol. 10619, pp. 183-195, 6th CCF International Conference, Dalian, China, 8/11/17. https://doi.org/10.1007/978-3-319-73618-1_16

A convolutional attention model for text classification. / Du, Jiachen; Gui, Lin; Xu, Ruifeng; He, Yulan.

English. 2018. p. 183-195 Chapter 16 (Natural Language Processing and Chinese Computing; Vol. 10619).

Research output: Chapter in Book/Report/Conference proceedingOther chapter contribution

TY - CHAP

T1 - A convolutional attention model for text classification

AU - Du, Jiachen

AU - Gui, Lin

AU - Xu, Ruifeng

AU - He, Yulan

PY - 2018/1/5

Y1 - 2018/1/5

N2 - Neural network models with attention mechanism have shown their efficiencies on various tasks. However, there is little research work on attention mechanism for text classification and existing attention model for text classification lacks of cognitive intuition and mathematical explanation. In this paper, we propose a new architecture of neural network based on the attention model for text classification. In particular, we show that the convolutional neural network (CNN) is a reasonable model for extracting attentions from text sequences in mathematics. We then propose a novel attention model base on CNN and introduce a new network architecture which combines recurrent neural network with our CNN-based attention model. Experimental results on five datasets show that our proposed models can accurately capture the salient parts of sentences to improve the performance of text classification.

AB - Neural network models with attention mechanism have shown their efficiencies on various tasks. However, there is little research work on attention mechanism for text classification and existing attention model for text classification lacks of cognitive intuition and mathematical explanation. In this paper, we propose a new architecture of neural network based on the attention model for text classification. In particular, we show that the convolutional neural network (CNN) is a reasonable model for extracting attentions from text sequences in mathematics. We then propose a novel attention model base on CNN and introduce a new network architecture which combines recurrent neural network with our CNN-based attention model. Experimental results on five datasets show that our proposed models can accurately capture the salient parts of sentences to improve the performance of text classification.

UR - https://www.scopus.com/record/display.uri?eid=2-s2.0-85041173594&origin=SingleRecordEmailAlert&dgcid=raven_sc_search_en_us_email&txGid=1a57e5c3a195f3d54162cee2ae1d99a8

UR - http://link.springer.com/10.1007/978-3-319-73618-1_16

U2 - 10.1007/978-3-319-73618-1_16

DO - 10.1007/978-3-319-73618-1_16

M3 - Other chapter contribution

T3 - Natural Language Processing and Chinese Computing

SP - 183

EP - 195

BT - English

ER -

Du J, Gui L, Xu R, He Y. A convolutional attention model for text classification. In English. 2018. p. 183-195. Chapter 16. (Natural Language Processing and Chinese Computing). https://doi.org/10.1007/978-3-319-73618-1_16