DOI: 10.14489/vkit.2020.03.pp.029-036

Вахромова Е. Ю., Бекетова И. В., Герасименко А. А., Горемычкин В. И., Кривошляпов В. П.
АЛГОРИТМ МЕЖЪЯЗЫКОВОГО НЕЧЕТКОГО ПОИСКА НА ОСНОВЕ ХЕШ-ВЕКТОРОВ ДЛЯ АВТОМАТИЧЕСКОГО СОПОСТАВЛЕНИЯ ИМЕННЫХ ГРУПП
(c. 29-36)

Аннотация. Предложен алгоритм межъязыкового нечеткого поиска на основе хешвекторов для автоматического сопоставления именных групп, принципиальная особенность которого – отказ от прямого перевода именных групп. Использован механизм хеширования именных групп с последующим отображением их в одно и то же скрытое векторное пространство, где построены вычислительные процедуры системы принятия решения.

Ключевые слова:  персональные данные; нечеткий поиск; кластерный анализ; хеширование; векторное пространство; базы данных.

Vakhromova E. Yu., Beketova I. V., Gerasimenko A. A., Goremychkin V. I., Krivoshlyapov V. P.
ALGORITHM OF CROSS LANGUAGE FUZZY SEARCH BASED ON HASH-VECTORS FOR AUTOMATIC COMPARISON OF PERSONAL NAMES
(pp. 29-36)

Abstract. The algorithm of cross language fuzzy search based on hash vectors for automatic matching of personal names is proposed. In the response mode for an input request, names in Latin spelling and a given value for the similarity measure, the algorithm determines the set of output Cyrillic names contained in the database of the information search system. The principal feature of the proposed algorithm is the rejection of the direct translation of personal names. Instead, the hashing mechanism of personal names is used, followed by mapping them into the same hidden vector space where the computational procedures of the decision-making system are built. In the process of research, it was solved a number of actual intermediate tasks. Thus, the decomposition algorithms of the explored database, the generation and clustering of the dictionary of basic morphemes are an instrument that is of independent value in solving the problem of automatically translating names from a foreign language, the translation rules of which are unknown – the socalled generalized transcription. After mapping names into a vector space, the matching operation is reduced to assessing the similarity between vectors. As a measure of similarity, several quantities were considered in the study. The most convenient measure of similarity is the cosine similarity, the critical value of which was obtained by plotting the FMR (False Match Rate) and FNMR (False Non-Match Rate) graphs. The developed algorithm is universal with respect to the languages used, that is, it does not depend on a specific alphabet. In the practical implementation of the developed algorithm, a series of experimental studies was carried out using a database containing more than 2.5 million names.

Keywords: Personal data; Fuzzy search; Cluster analysis; Hashing; Vector space; Databases.