Improving word embeddings projection for Turkish hypernym extraction

Abstract

Corpus-driven approaches can automatically explore is-a relations between the word pairs from corpus. Thisproblem is also called hypernym extraction. Formerly, lexico-syntactic patterns have been used to solve hypernymrelations. The language-specific syntactic rules have been manually crafted to build the patterns. On the other hand,recent studies have applied distributional approaches to word semantics. They extracted the semantic relations relyingon the idea that similar words share similar contexts. Former distributional approaches have applied one-hot bag-ofword (BOW) encoding. The dimensionality problem of BOW has been solved by various neural network approaches,which represent words in very short and dense vectors, or word embeddings. In this study, we used word embeddingsrepresentation and employed the optimized projection algorithm to solve the hypernym problem. The supervisedarchitecture learns a mapping function so that the embeddings (or vectors) of word pairs that are in hypernym relationscan be projected to each other. In the training phase, the architecture first learns the embeddings of words and theprojection function from a list of word pairs. In the test phase, the projection function maps the embeddings of a givenword to a point that is the closest to its hypernym. We utilized the deep learning optimization methods to optimizethe model and improve the performances by tuning hyperparameters. We discussed our results by carrying out manyexperiments based on cross-validation. We also addressed problem-specific loss function, monitored hyperparameters, andevaluated the results with respect to different settings. Finally, we successfully showed that our approach outperformedbaseline functions and other studies in the Turkish language.