Using the determining the similarity of words method to evaluate text vectorization algorithms
Abstract
Using the determining the similarity of words method to evaluate text vectorization algorithms
Incoming article date: 26.05.2024The article provides a brief description of the existing methods of vectorization of texts in natural language. The evaluation is described by the method of determining the similarity of words. A comparative analysis of the operation of several vectorizer models is carried out. The process of selecting data for evaluation is described. The results of evaluating the performance of the models are compared.
Keywords: natural language processing, vectorization, word-form embedding, semantic similarity, correlation