A rule extraction study from SVM on sentiment analysis

Bologna, Guido (School of Engineering, Architecture and Landscape (hepia), HES-SO // University of Applied Sciences Western Switzerland ; Department of Computer Science, University of Geneva, Carouge, Switzerland) ; Hayashi, Yoichi (Department of Computer Science, Meiji University, Tama-ku, Kawasaki Kanagawa, Japan)

A natural way to determine the knowledge embedded within connectionist models is to generate symbolic rules. Nevertheless, extracting rules from Multi Layer Perceptrons (MLPs) is NP-hard. With the advent of social networks, techniques applied to Sentiment Analysis show a growing interest, but rule extraction from connectionist models in this context has been rarely performed because of the very high dimensionality of the input space. To fill the gap we present a case study on rule extraction from ensembles of Neural Networks and Support Vector Machines (SVMs), the purpose being the characterization of the complexity of the rules on two particular Sentiment Analysis problems. Our rule extraction method is based on a special Multi Layer Perceptron architecture for which axis-parallel hyperplanes are precisely located. Two datasets representing movie reviews are transformed into Bag-of-Words vectors and learned by ensembles of neural networks and SVMs. Generated rules from ensembles of MLPs are less accurate and less complex than those extracted from SVMs. Moreover, a clear trade-off appears between rules’ accuracy, complexity and covering. For instance, if rules are too complex, less complex rules can be re-extracted by sacrificing to some extent their accuracy. Finally, rules can be viewed as feature detectors in which very often only one word must be present and a longer list of words must be absent.

Article Type:
Ingénierie et Architecture
HEPIA - Genève
inIT - Institut d'Ingénierie Informatique et des Télécommunications
19 p.
Published in:
Big Data and Cognitive Computing
Numeration (vol. no.):
2018, vol. 2, no. 1, article no. 6
Appears in Collection:

 Record created 2020-08-25, last modified 2020-10-27

Download fulltext

Rate this document:

Rate this document:
(Not yet reviewed)