A Comparison of Term Weighting Schemes for Text Classification and Sentiment Analysis with a Supervised Variant of tf.idf


Giacomo Domeniconi, Gianluca Moro, Roberto Pasolini, Claudio Sartori

Data Management Technologies and Applications: 4th International Conference, DATA 2015, Colmar, France, July 20-22, 2015, Revised Selected Papers, pages 39-59
Communications in Computer and Information Science 584,  2016
Markus Helfert, Andreas Holzinger, Orlando Belo, Chiara Francalanci (eds.)

In text analysis tasks like text classification and sentiment analysis, the careful choice of term weighting schemes can have an important impact on the effectiveness. Classic unsupervised schemes are based solely on the distribution of terms across documents, while newer supervised ones leverage the knowledge of membership of training documents to categories; these latter ones are often specifically tailored for either topic or sentiment classification. We propose here a supervised variant of the well-known tf.idf scheme, where the idf factor is computed without considering documents within the category under analysis, so that terms frequently appearing only within it are not penalized. The importance of these terms is further boosted in a second variant inspired by relevance frequency. We performed extensive experiments to compare these novel schemes to known ones, observing top performances in text categorization by topic and satisfactory results in sentiment classification.

Tags:

Publication

— authors

Giacomo Domeniconi, Gianluca Moro, Roberto Pasolini, Claudio Sartori

— editors

Markus Helfert, Andreas Holzinger, Orlando Belo, Chiara Francalanci

— status

published

— sort

paper in proceedings

Venue

— volume

Data Management Technologies and Applications: 4th International Conference, DATA 2015, Colmar, France, July 20-22, 2015, Revised Selected Papers

— series

Communications in Computer and Information Science

— volume

584

— pages

39-59

— publication date

2016

URLs

original page

Identifiers

— DOI

10.1007/978-3-319-30162-4_4

— print ISSN

978-3-319-30162-4

BibTeX

— BibTeX category
article

Partita IVA: 01131710376 - Copyright © 2008-2022 APICe@DISI Research Group - PRIVACY