A convolutional attentional neural network for sentiment classification

Jiachen Du, Lin Gui, Yulan He, Ruifeng Xu

Research output: Contribution to conferenceOther

Abstract

Neural network models with attention mechanism have shown their efficiencies on various tasks. However, there is little research work on attention mechanism for text classification and existing attention model for text classification lacks of cognitive intuition and mathematical explanation. In this paper, we propose a new architecture of neural network based on the attention model for text classification. In particular, we show that the convolutional neural network (CNN) is a reasonable model for extracting attentions from text sequences in mathematics. We then propose a novel attention model base on CNN and introduce a new network architecture which combines recurrent neural network with our CNN-based attention model. Experimental results on five datasets show that our proposed models can accurately capture the salient parts of sentences to improve the performance of text classification.
Original languageEnglish
Pages445-450
DOIs
Publication statusE-pub ahead of print - 1 Mar 2018
Event2017 International Conference on Security, Pattern Analysis, and Cybernetics (SPAC) - Shenzhen, China
Duration: 15 Dec 201717 Dec 2017

Conference

Conference2017 International Conference on Security, Pattern Analysis, and Cybernetics (SPAC)
Period15/12/1717/12/17

Fingerprint

Neural networks
Recurrent neural networks
Network architecture

Bibliographical note

© 2018 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.

Funding: National Natural Science Foundation of China 61370165, U1636103, 61632011, Shenzhen Foundational Research Funding
JCYJ20150625142543470, 20170307150024907, Guangdong
Provincial Engineering Technology Research Center for Data
Science 2016KF09.

Cite this

Du, J., Gui, L., He, Y., & Xu, R. (2018). A convolutional attentional neural network for sentiment classification. 445-450. 2017 International Conference on Security, Pattern Analysis, and Cybernetics (SPAC), . https://doi.org/10.1109/SPAC.2017.8304320
Du, Jiachen ; Gui, Lin ; He, Yulan ; Xu, Ruifeng. / A convolutional attentional neural network for sentiment classification. 2017 International Conference on Security, Pattern Analysis, and Cybernetics (SPAC), .
@conference{54c59348fa6b40cca83e0e8ab9468d3e,
title = "A convolutional attentional neural network for sentiment classification",
abstract = "Neural network models with attention mechanism have shown their efficiencies on various tasks. However, there is little research work on attention mechanism for text classification and existing attention model for text classification lacks of cognitive intuition and mathematical explanation. In this paper, we propose a new architecture of neural network based on the attention model for text classification. In particular, we show that the convolutional neural network (CNN) is a reasonable model for extracting attentions from text sequences in mathematics. We then propose a novel attention model base on CNN and introduce a new network architecture which combines recurrent neural network with our CNN-based attention model. Experimental results on five datasets show that our proposed models can accurately capture the salient parts of sentences to improve the performance of text classification.",
author = "Jiachen Du and Lin Gui and Yulan He and Ruifeng Xu",
note = "{\circledC} 2018 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. Funding: National Natural Science Foundation of China 61370165, U1636103, 61632011, Shenzhen Foundational Research Funding JCYJ20150625142543470, 20170307150024907, Guangdong Provincial Engineering Technology Research Center for Data Science 2016KF09.; 2017 International Conference on Security, Pattern Analysis, and Cybernetics (SPAC) ; Conference date: 15-12-2017 Through 17-12-2017",
year = "2018",
month = "3",
day = "1",
doi = "10.1109/SPAC.2017.8304320",
language = "English",
pages = "445--450",

}

Du, J, Gui, L, He, Y & Xu, R 2018, 'A convolutional attentional neural network for sentiment classification' 2017 International Conference on Security, Pattern Analysis, and Cybernetics (SPAC), 15/12/17 - 17/12/17, pp. 445-450. https://doi.org/10.1109/SPAC.2017.8304320

A convolutional attentional neural network for sentiment classification. / Du, Jiachen; Gui, Lin; He, Yulan; Xu, Ruifeng.

2018. 445-450 2017 International Conference on Security, Pattern Analysis, and Cybernetics (SPAC), .

Research output: Contribution to conferenceOther

TY - CONF

T1 - A convolutional attentional neural network for sentiment classification

AU - Du, Jiachen

AU - Gui, Lin

AU - He, Yulan

AU - Xu, Ruifeng

N1 - © 2018 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. Funding: National Natural Science Foundation of China 61370165, U1636103, 61632011, Shenzhen Foundational Research Funding JCYJ20150625142543470, 20170307150024907, Guangdong Provincial Engineering Technology Research Center for Data Science 2016KF09.

PY - 2018/3/1

Y1 - 2018/3/1

N2 - Neural network models with attention mechanism have shown their efficiencies on various tasks. However, there is little research work on attention mechanism for text classification and existing attention model for text classification lacks of cognitive intuition and mathematical explanation. In this paper, we propose a new architecture of neural network based on the attention model for text classification. In particular, we show that the convolutional neural network (CNN) is a reasonable model for extracting attentions from text sequences in mathematics. We then propose a novel attention model base on CNN and introduce a new network architecture which combines recurrent neural network with our CNN-based attention model. Experimental results on five datasets show that our proposed models can accurately capture the salient parts of sentences to improve the performance of text classification.

AB - Neural network models with attention mechanism have shown their efficiencies on various tasks. However, there is little research work on attention mechanism for text classification and existing attention model for text classification lacks of cognitive intuition and mathematical explanation. In this paper, we propose a new architecture of neural network based on the attention model for text classification. In particular, we show that the convolutional neural network (CNN) is a reasonable model for extracting attentions from text sequences in mathematics. We then propose a novel attention model base on CNN and introduce a new network architecture which combines recurrent neural network with our CNN-based attention model. Experimental results on five datasets show that our proposed models can accurately capture the salient parts of sentences to improve the performance of text classification.

UR - http://ieeexplore.ieee.org/document/8304320/

U2 - 10.1109/SPAC.2017.8304320

DO - 10.1109/SPAC.2017.8304320

M3 - Other

SP - 445

EP - 450

ER -

Du J, Gui L, He Y, Xu R. A convolutional attentional neural network for sentiment classification. 2018. 2017 International Conference on Security, Pattern Analysis, and Cybernetics (SPAC), . https://doi.org/10.1109/SPAC.2017.8304320