A Comparison of Term Weighting Schemes for Text Classification and Sentiment Analysis with a Supervised Variant of tf.idf


Giacomo Domeniconi, Gianluca Moro, Roberto Pasolini, Claudio Sartori

In text analysis tasks like text classification and sentiment analysis, the careful choice of term weighting schemes can have an important impact on the effectiveness. Classic unsupervised schemes are based solely on the distribution of terms across documents, while newer supervised ones leverage the knowledge of membership of training documents to categories; these latter ones are often specifically tailored for either topic or sentiment classification. We propose here a supervised variant of the well-known tf.idf scheme, where the idf factor is computed without considering documents within the category under analysis, so that terms frequently appearing only within it are not penalized. The importance of these terms is further boosted in a second variant inspired by relevance frequency. We performed extensive experiments to compare these novel schemes to known ones, observing top performances in text categorization by topic and satisfactory results in sentiment classification.

Data Management Technologies and Applications: 4th International Conference, DATA 2015, Colmar, France, July 20-22, 2015, Revised Selected Papers, Communications in Computer and Information Science 584, pages 39-59, 2016.
Markus Helfert, Andreas Holzinger, Orlando Belo, Chiara Francalanci (eds.)

Tags:
    

Publication

A Comparison of Term Weighting Schemes for Text Classification and Sentiment Analysis with a Supervised Variant of tf.idf

— status

published  

— authors

Giacomo Domeniconi, Gianluca Moro, Roberto Pasolini, Claudio Sartori

— editors

Markus Helfert, Andreas Holzinger, Orlando Belo, Chiara Francalanci

URLs & IDs

original page

— DOI

10.1007/978-3-319-30162-4_4

— print ISSN

978-3-319-30162-4

BibTeX

— BibTeX category
article

Partita IVA: 01131710376 - Copyright © 2008-2021 APICe@DISI Research Group - PRIVACY