Evaluating features for rhetorical structure classification in scientific abstracts

Abstract

Rhetorical structure classification is a NLP task in which we want to identify the rhetorical components of a discourse and its relationships. In this work, we aimed at automatically identifying propositions at the sentential level that make up the rhetorical structure of scientific abstracts. Specifically, the purpose was to evaluate the impact of different sets of attributes on the implementation of rhetorical classifiers for scientific abstracts written in Portuguese. For this, we used superficial features (extracted as TF-IDF values and selected with the $\chi^2$ test), morphosyntactic features (implemented by the AZPort classifier) and features extracted from \textit {word embeddings} models (Word2Vec, Wang2Vec and GloVe, all of them previously trained). These sets of features, as well as its combinations, were used for the training of the following supervised learning classifiers: Support Vector Machines, Naive Bayes, K-Nearest Neighbors, Decision Trees and Conditional Random Fields (CRF). They were trained and tested through cross-validation on three \textit{corpora} composed by abstracts of theses and dissertations. The best result, $94\%$ of F1, was obtained by the CRF classifier with the following combinations of features: (i) Wang2Vec--Skip-gram of $100$ dimension with the features from AZPort; (ii) TF-IDF, AZPort and \textit{embeddings} extracted with the Word2Vec--Skip-gram and GloVe models of dimensions $1000$ and $300$, respectively. From the results, we concluded that the AZPort features were fundamental for the performance of the CRF and that the combination with \textit{word embeddings} proved valid.