Skip to main content
×
×
Home

Aspect opinion expression and rating prediction via LDA–CRF hybrid

  • ABHISHEK LADDHA (a1) and ARJUN MUKHERJEE (a2)
Abstract

In this paper, we study the problem of aspect-based sentiment analysis. Our model simultaneously extracts aspect-specific opinion expressions and determines the rating for each aspect in reviews. Previous works have mainly focused on the problem of opinion phrase extraction and aspect rating prediction in a pipelined manner and are not able to capture the dependencies of aspect opinion expression on aspect rating and vice-versa. They are also unable to discover aspect-specific opinion expressions and their associated rating scores. We present a joint modelling approach to extract aspect-specific sentiment expression and aspect rating prediction simultaneously. This paper proposes a novel LDA–CRF hybrid model which employs discriminative conditional random field component for phrase extraction, a regression component for rating prediction and a generative component for grouping aspect–sentiment expressions (aspect-specific opinion expressions) into coherent topics. To show the effectiveness of our approach, we evaluate the performance of the model on both task: (i) aspect-specific opinion expressions and (ii) rating prediction on the dataset of hotel and restaurant reviews from TripAdvisor.com. Experimental results show that both task potentially reinforce each other and joint modeling outperformed state-of-the-art baselines for each individual tasks.

Copyright
References
Hide All
Bengio, S., Vinyals, O., Jaitly, N., and Shazeer, N. 2015. Scheduled sampling for sequence prediction with recurrent neural networks. In Proceedings of the 28th International Conference on Advances in Neural Information Processing Systems, pp. 1171–9.
Breck, E., Choi, Y., and Cardie, C. 2007. Identifying expressions of opinion in context. In IJCAI, vol. 7, pp. 2683–8.
Brody, S., and Elhadad, N. 2010. An unsupervised aspect-sentiment model for online reviews. In Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, Association for Computational Linguistics, pp. 804–12.
Chang, C.-C., and Lin, C.-J. 2011. LIBSVM: a library for support vector machines. ACM Transactions on Intelligent Systems and Technology 2. http://www.csie.ntu.edu.tw/~cjlin/libsvm.
Choi, Y., Breck, E., and Cardie, C. 2006. Joint extraction of entities and relations for opinion recognition. In Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics, pp. 431–9.
Choi, Y., Cardie, C., Riloff, E., and Patwardhan, S. 2005. Identifying sources of opinions with conditional random fields and extraction patterns. In Proceedings of the Conference on Human Language Technology and Empirical Methods in Natural Language Processing, Association for Computational Linguistics, pp. 355–62.
Dave, K., Lawrence, S., and Pennock, D. M. 2003. Mining the peanut gallery: opinion extraction and semantic classification of product reviews. In Proceedings of the 12th International Conference on World Wide Web, ACM, pp. 519–28.
Diao, Q., Qiu, M., Wu, C.-Y., Smola, A. J., Jiang, J., and Wang, C. 2014. Jointly modeling aspects, ratings and sentiments for movie recommendation (jmars). In Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, ACM, pp. 193–202.
Diebolt, J., and Ip, E. 1996. Markov chain monte carlo in practice.
El-Kishky, A., Song, Y., Wang, C., Voss, C. R., and Han, J., 2014. Scalable topical phrase mining from text corpora. Proceedings of the VLDB Endowment 8 (3): 305–16.
Fei, G., Chen, Z., and Liu, B. 2014. Review topic discovery with phrases using the pólya urn model. In COLING, pp. 667–76.
Fu, X., Wu, H., and Cui, L. 2016. Topic sentiment joint model with word embeddings. In DMNLP@ PKDD/ECML, pp. 41–8.
Fu, X., Yang, K., Huang, J. Z., and Cui, L., 2015. Dynamic non-parametric joint sentiment topic mixture model. Knowledge-Based Systems 82 : 102–14.
Hu, M., and Liu, B. 2004. Mining and summarizing customer reviews. In Proceedings of the 10th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, ACM, pp. 168–77.
Huang, Z., Xu, W., and Yu, K. 2015. Bidirectional lstm-crf models for sequence tagging. arXiv preprint, arXiv:1508.01991.
Irsoy, O., and Cardie, C. 2014. Opinion mining with deep recurrent neural networks. In EMNLP, pp. 720–8.
Jakob, N., and Gurevych, I. 2010. Extracting opinion targets in a single-and cross-domain setting with conditional random fields. In Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics, pp. 1035–45.
Jo, Y. and Oh, A. H. 2011. Aspect and sentiment unification model for online review analysis. In Proceedings of the 4th ACM International Conference on Web Search and Data Mining, ACM, pp. 815–24.
Johansson, R., and Moschitti, A., 2011. Extracting opinion expressions and their polarities: exploration of pipelines and joint models. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies: short papers, vol. 2, Association for Computational Linguistics, pp. 101–6.
Kim, S.-M., and Hovy, E. 2006. Extracting opinions, opinion holders, and topics expressed in online news media text. In Proceedings of the Workshop on Sentiment and Subjectivity in Text, Association for Computational Linguistics, pp. 1–8.
Kobayashi, N., Inui, K., and Matsumoto, Y., 2007. Extracting aspect-evaluation and aspect-of relations in opinion mining. In EMNLP-CoNLL, vol. 7, Citeseer, pp. 1065–74.
Kudo, T. 2009. Crf++: Yet another crf toolkit [ol].
Lafferty, J., McCallum, A., and Pereira, F. 2001. Conditional random fields: probabilistic models for segmenting and labeling sequence data. In Proceedings of the 18th International Conference on Machine Learning, ICML, vol. 1, pp. 282–9.
Lakkaraju, H., Bhattacharyya, C., Bhattacharya, I., and Merugu, S. 2011. Exploiting coherence for the simultaneous discovery of latent facets and associated sentiments. In SDM, SIAM, pp. 498–509.
Lakkaraju, H., Socher, R., and Manning, C. 2014. Aspect specific sentiment analysis using hierarchical deep learning. In NIPS Workshop on Deep Learning and Representation Learning.
Li, F., Huang, M., and Zhu, X. 2010. Sentiment analysis with global topics and local dependency. In AAAI, vol. 10, pp. 1371–6.
Lin, C., and He, Y. 2009. Joint sentiment/topic model for sentiment analysis. In Proceedings of the 18th ACM Conference on Information and Lnowledge Management, ACM, pp. 375–84.
Lindsey, R. V., Headden, W. P. III, and Stipicevic, M. J. 2012. A phrase-discovering topic model using hierarchical pitman-yor processes. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, Association for Computational Linguistics, pp. 214–22.
Liu, B. and Zhang, L. 2012. A survey of opinion mining and sentiment analysis. In Mining Text Data, pp. 415–63. Springer, pp. 415–63.
Liu, J., Shang, J., Wang, C., Ren, X., and Han, J. 2015. Mining quality phrases from massive text corpora. In Proceedings of the 2015 ACM SIGMOD International Conference on Management of Data, ACM, pp. 1729–44.
Liu, P., Joty, S. R., and Meng, H. M. 2015. Fine-grained opinion mining with recurrent neural networks and word embeddings. In EMNLP, pp. 1433–43.
Lu, B., Ott, M., Cardie, C., and Tsou, B. K. 2011. Multi-aspect sentiment analysis with topic models. In Proceedings of the IEEE 11th International Conference on Data Mining Workshops, IEEE, pp. 81–8.
Lu, Y., Zhai, C., and Sundaresan, N. 2009. Rated aspect summarization of short comments. In Proceedings of the 18th International Conference on World Wide Web, ACM, pp. 131–40.
Mahmoud, H. 2008. Pólya urn Models. CRC press.
Mcauliffe, J. D., and Blei, D. M. 2008. Supervised topic models. In Advances in Neural Information Processing Systems, pp. 121–8.
Mei, Q., Ling, X., Wondra, M., Su, H., and Zhai, C. 2007. Topic sentiment mixture: modeling facets and opinions in weblogs. In Proceedings of the 16th International Conference on World Wide Web, ACM, pp. 171–80.
Mimno, D. and McCallum, A. 2012. Topic models conditioned on arbitrary features with dirichlet-multinomial regression. arXiv preprint, arXiv:1206.3278.
Mimno, D., Wallach, H. M., Talley, E., Leenders, M., and McCallum, A. 2011. Optimizing semantic coherence in topic models. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics, pp. 262–72.
Moghaddam, S., and Ester, M. 2011. Ilda: interdependent lda model for learning latent aspects and their ratings from online product reviews. In Proceedings of the 34th International ACM SIGIR Conference on Research and Development in Information Retrieval, ACM, pp. 665–74.
Moghaddam, S., and Ester, M. 2013. The flda model for aspect-based opinion mining: addressing the cold start problem. In Proceedings of the 22nd International Conference on World Wide Web, International World Wide Web Conferences Steering Committee, pp. 909–18.
Mukherjee, A., and Liu, B., 2012. Aspect extraction through semi-supervised modeling. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Long Papers, vol. 1, Association for Computational Linguistics, pp. 339–48.
Nguyen, T. H., and Shirai, K. 2015. Phrasernn: phrase recursive neural network for aspect-based sentiment analysis. In EMNLP, pp. 2509–14.
Pennington, J., Socher, R., and Manning, C. D. 2014. Glove: global vectors for word representation. In EMNLP, vol. 14, pp. 1532–43.
Pontiki, M., et al. 2016. Semeval-2016 task 5: aspect based sentiment analysis. In ProWorkshop on Semantic Evaluation (SemEval-2016), Association for Computational Linguistics, pp. 19–30.
Popescu, A.-M., and Etzioni, O. 2007. Extracting product features and opinions from reviews. In Natural Language Processing and Text Mining, Springer, pp. 9–28.
Poria, S., Cambria, E., and Gelbukh, A. 2016. Aspect extraction for opinion mining with a deep convolutional neural network. Knowledge-Based Systems 108, 42–9.
Poria, S., Chaturvedi, I., Cambria, E., and Bisio, F. 2016. Sentic lda: improving on lda with semantic similarity for aspect-based sentiment analysis. In Proceedings of the International Joint Conference on Neural Networks (IJCNN), pp. 4465–73.
Sauper, C., and Barzilay, R., 2013. Automatic aggregation by joint modeling of aspects and values. Journal of Artificial Intelligence Research 46 : 89127.
Sauper, C., Haghighi, A., and Barzilay, R., 2011. Content models with attitude. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, vol. 1, Association for Computational Linguistics, pp. 350–8.
Snyder, B., and Barzilay, R. 2007. Multiple aspect ranking using the good grief algorithm. In HLT-NAACL, pp. 300–7.
Somasundaran, S., Namata, G., Wiebe, J., and Getoor, L., 2009. Supervised and unsupervised methods in employing discourse relations for improving opinion polarity classification. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, vol. 1, Association for Computational Linguistics, pp. 170–9.
Tang, D., Qin, B., Feng, X., and Liu, T. 2015. Effective lstms for target-dependent sentiment classification. arXiv preprint, arXiv:1512.01100.
Tang, D., Qin, B., and Liu, T. 2016. Aspect level sentiment classification with deep memory network. arXiv preprint, arXiv:1605.08900.
Titov, I., and McDonald, R. T., 2008. A joint model of text and aspect ratings for sentiment summarization. In ACL, vol. 8, Citeseer, pp. 308–16.
Wang, H. 2015. Sentiment-Aligned Topic Models for Product Aspect Rating Prediction. Ph D Thesis, Applied Sciences, School of Computing Science.
Wang, H., Lu, Y., and Zhai, C. 2010. Latent aspect rating analysis on review text data: a rating regression approach. In Proceedings of the 16th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, ACM, pp. 783–92.
Wang, H., Lu, Y., and Zhai, C. 2011. Latent aspect rating analysis without aspect keyword supervision. In Proceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, ACM, pp. 618–26.
Wang, W., Pan, S. J., Dahlmeier, D., and Xiao, X. 2016. Recursive neural conditional random fields for aspect-based sentiment analysis. arXiv preprint, arXiv:1603.06679.
Wang, X., McCallum, A., and Wei, X. 2007. Topical n-grams: phrase and topic discovery, with an application to information retrieval. In Proceedings of the 7th IEEE International Conference on Data Mining, ICDM 2007, pp. 697–702.
Wang, Y., Huang, M., Zhao, L., and Zhu, X. 2016. Attention-based lstm for aspect-level sentiment classification. In EMNLP, pp. 606–15.
Wiebe, J., Wilson, T., and Cardie, C., 2005. Annotating expressions of opinions and emotions in language. Language Resources and Evaluation 39 (2–3): 165210.
Wu, Y., Zhang, Q., Huang, X., and Wu, L., 2009. Phrase dependency parsing for opinion mining. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, vol. 3, Association for Computational Linguistics, pp. 1533–41.
Xianghua, F., Guo, L., Yanyan, G., and Zhiqiang, W., 2013. Multi-aspect sentiment analysis for chinese online social reviews based on topic modeling and hownet lexicon. Knowledge-Based Systems 37 : 186–95.
Xu, Y., Lin, T., Lam, W., Zhou, Z., Cheng, H., and So, A. M.-C. 2014. Latent aspect mining via exploring sparsity and intrinsic information. In Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management, ACM, pp. 879–88.
Yang, B., and Cardie, C. 2012. Extracting opinion expressions with semi-markov conditional random fields. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, Association for Computational Linguistics, pp. 1335–45.
Yang, B., and Cardie, C. 2013. Joint inference for fine-grained opinion extraction. In ACL (1), pp. 1640–9.
Yang, B., and Cardie, C., 2014. Joint modeling of opinion expression extraction and attribute classification. Transactions of the Association for Computational Linguistics 2 : 505–16.
Yin, Y., Wei, F., Dong, L., Xu, K., Zhang, M., and Zhou, M. 2016. Unsupervised word and dependency path embeddings for aspect term extraction. arXiv preprint, arXiv:1605.07843.
Zhao, W. X., Jiang, J., He, J., Song, Y., Achananuparp, P., Lim, E.-P., and Li, X., 2011. Topical keyphrase extraction from twitter. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, vol. 1, Association for Computational Linguistics, pp. 379–88.
Zhao, W. X., Jiang, J., Yan, H., and Li, X. 2010. Jointly modeling aspects and opinions with a maxent-lda hybrid. In Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics, pp. 56–65.
Zhu, J., Ahmed, A., and Xing, E. P., 2012. Medlda: maximum margin supervised topic models. Journal of Machine Learning Research 13 : 2237–78.
Zhuang, L., Jing, F., and Zhu, X.-Y. 2006. Movie review mining and summarization. In Proceedings of the 15th ACM International Conference on Information and Knowledge Management, ACM, pp. 43–50.
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

Natural Language Engineering
  • ISSN: 1351-3249
  • EISSN: 1469-8110
  • URL: /core/journals/natural-language-engineering
Please enter your name
Please enter a valid email address
Who would you like to send this to? *
×

Metrics

Full text views

Total number of HTML views: 5
Total number of PDF views: 50 *
Loading metrics...

Abstract views

Total abstract views: 267 *
Loading metrics...

* Views captured on Cambridge Core between 22nd April 2018 - 19th August 2018. This data will be updated every 24 hours.