Avaliação no Desafio de Identificação de Personagens

Abstract

The first edition of the Character Identification Challenge (DIP) was a joint evaluation of computational solutions for the identification of characters in literary texts, as well as the extraction of characteristics and relationships of these characters. For this evaluation, it was necessary to define an evaluation methodology, including the selection of appropriate metrics for the problem of identifying characters in literary texts. This article surveys the evaluation methods employed in character identification in literary works and presents the concrete choices done in DIP. These choices resulted in the definition of the DIP evaluation methodology. The use of the proposed evaluation methodology is illustrated by the evaluation of the candidate solution submitted to DIP. We end the paper with some critical remarks and suggestions for improvement.