Learning user and product distributed representations using a sequence model for sentiment analysis

Tao Chen, Ruifeng Xu*, Yulan He, Yunqing Xia, Xuan Wang

*Corresponding author for this work

Research output: Contribution to journalArticle

Abstract

In product reviews, it is observed that the distribution of polarity ratings over reviews written by different users or evaluated based on different products are often skewed in the real world. As such, incorporating user and product information would be helpful for the task of sentiment classification of reviews. However, existing approaches ignored the temporal nature of reviews posted by the same user or evaluated on the same product. We argue that the temporal relations of reviews might be potentially useful for learning user and product embedding and thus propose employing a sequence model to embed these temporal relations into user and product representations so as to improve the performance of document-level sentiment analysis. Specifically, we first learn a distributed representation of each review by a one-dimensional convolutional neural network. Then, taking these representations as pretrained vectors, we use a recurrent neural network with gated recurrent units to learn distributed representations of users and products. Finally, we feed the user, product and review representations into a machine learning classifier for sentiment classification. Our approach has been evaluated on three large-scale review datasets from the IMDB and Yelp. Experimental results show that: (1) sequence modeling for the purposes of distributed user and product representation learning can improve the performance of document-level sentiment classification; (2) the proposed approach achieves state-of-The-Art results on these benchmark datasets.

Original languageEnglish
Pages (from-to)34-44
Number of pages11
JournalIEEE Computational Intelligence Magazine
Volume11
Issue number3
Early online date18 Jul 2016
DOIs
Publication statusPublished - Aug 2016

Fingerprint

Sentiment Analysis
Model
Learning
Recurrent neural networks
Review
Recurrent Neural Networks
Polarity
Learning systems
Classifiers
Machine Learning
Neural networks
Classifier
Neural Networks
Benchmark
Unit

Bibliographical note

© 2016 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.

Cite this

Chen, T., Xu, R., He, Y., Xia, Y., & Wang, X. (2016). Learning user and product distributed representations using a sequence model for sentiment analysis. IEEE Computational Intelligence Magazine, 11(3), 34-44. https://doi.org/10.1109/MCI.2016.2572539
Chen, Tao ; Xu, Ruifeng ; He, Yulan ; Xia, Yunqing ; Wang, Xuan. / Learning user and product distributed representations using a sequence model for sentiment analysis. In: IEEE Computational Intelligence Magazine. 2016 ; Vol. 11, No. 3. pp. 34-44.
@article{26fd734a969f42e99e34fb0fe23efebe,
title = "Learning user and product distributed representations using a sequence model for sentiment analysis",
abstract = "In product reviews, it is observed that the distribution of polarity ratings over reviews written by different users or evaluated based on different products are often skewed in the real world. As such, incorporating user and product information would be helpful for the task of sentiment classification of reviews. However, existing approaches ignored the temporal nature of reviews posted by the same user or evaluated on the same product. We argue that the temporal relations of reviews might be potentially useful for learning user and product embedding and thus propose employing a sequence model to embed these temporal relations into user and product representations so as to improve the performance of document-level sentiment analysis. Specifically, we first learn a distributed representation of each review by a one-dimensional convolutional neural network. Then, taking these representations as pretrained vectors, we use a recurrent neural network with gated recurrent units to learn distributed representations of users and products. Finally, we feed the user, product and review representations into a machine learning classifier for sentiment classification. Our approach has been evaluated on three large-scale review datasets from the IMDB and Yelp. Experimental results show that: (1) sequence modeling for the purposes of distributed user and product representation learning can improve the performance of document-level sentiment classification; (2) the proposed approach achieves state-of-The-Art results on these benchmark datasets.",
author = "Tao Chen and Ruifeng Xu and Yulan He and Yunqing Xia and Xuan Wang",
note = "{\circledC} 2016 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.",
year = "2016",
month = "8",
doi = "10.1109/MCI.2016.2572539",
language = "English",
volume = "11",
pages = "34--44",
number = "3",

}

Chen, T, Xu, R, He, Y, Xia, Y & Wang, X 2016, 'Learning user and product distributed representations using a sequence model for sentiment analysis', IEEE Computational Intelligence Magazine, vol. 11, no. 3, pp. 34-44. https://doi.org/10.1109/MCI.2016.2572539

Learning user and product distributed representations using a sequence model for sentiment analysis. / Chen, Tao; Xu, Ruifeng; He, Yulan; Xia, Yunqing; Wang, Xuan.

In: IEEE Computational Intelligence Magazine, Vol. 11, No. 3, 08.2016, p. 34-44.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Learning user and product distributed representations using a sequence model for sentiment analysis

AU - Chen, Tao

AU - Xu, Ruifeng

AU - He, Yulan

AU - Xia, Yunqing

AU - Wang, Xuan

N1 - © 2016 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.

PY - 2016/8

Y1 - 2016/8

N2 - In product reviews, it is observed that the distribution of polarity ratings over reviews written by different users or evaluated based on different products are often skewed in the real world. As such, incorporating user and product information would be helpful for the task of sentiment classification of reviews. However, existing approaches ignored the temporal nature of reviews posted by the same user or evaluated on the same product. We argue that the temporal relations of reviews might be potentially useful for learning user and product embedding and thus propose employing a sequence model to embed these temporal relations into user and product representations so as to improve the performance of document-level sentiment analysis. Specifically, we first learn a distributed representation of each review by a one-dimensional convolutional neural network. Then, taking these representations as pretrained vectors, we use a recurrent neural network with gated recurrent units to learn distributed representations of users and products. Finally, we feed the user, product and review representations into a machine learning classifier for sentiment classification. Our approach has been evaluated on three large-scale review datasets from the IMDB and Yelp. Experimental results show that: (1) sequence modeling for the purposes of distributed user and product representation learning can improve the performance of document-level sentiment classification; (2) the proposed approach achieves state-of-The-Art results on these benchmark datasets.

AB - In product reviews, it is observed that the distribution of polarity ratings over reviews written by different users or evaluated based on different products are often skewed in the real world. As such, incorporating user and product information would be helpful for the task of sentiment classification of reviews. However, existing approaches ignored the temporal nature of reviews posted by the same user or evaluated on the same product. We argue that the temporal relations of reviews might be potentially useful for learning user and product embedding and thus propose employing a sequence model to embed these temporal relations into user and product representations so as to improve the performance of document-level sentiment analysis. Specifically, we first learn a distributed representation of each review by a one-dimensional convolutional neural network. Then, taking these representations as pretrained vectors, we use a recurrent neural network with gated recurrent units to learn distributed representations of users and products. Finally, we feed the user, product and review representations into a machine learning classifier for sentiment classification. Our approach has been evaluated on three large-scale review datasets from the IMDB and Yelp. Experimental results show that: (1) sequence modeling for the purposes of distributed user and product representation learning can improve the performance of document-level sentiment classification; (2) the proposed approach achieves state-of-The-Art results on these benchmark datasets.

UR - http://www.scopus.com/inward/record.url?scp=84979656415&partnerID=8YFLogxK

U2 - 10.1109/MCI.2016.2572539

DO - 10.1109/MCI.2016.2572539

M3 - Article

AN - SCOPUS:84979656415

VL - 11

SP - 34

EP - 44

IS - 3

ER -

Chen T, Xu R, He Y, Xia Y, Wang X. Learning user and product distributed representations using a sequence model for sentiment analysis. IEEE Computational Intelligence Magazine. 2016 Aug;11(3):34-44. https://doi.org/10.1109/MCI.2016.2572539