SavedAdd your journal

Journals / Anadolu Üniversitesi Bilim ve Teknoloji Dergisi :A-Uygulamalı Bilimler ve Mühendislik / 2017 / Cilt: 18 - Sayı: 2

STOP WORD DETECTION AS A BINARY CLASSIFICATION PROBLEM

Bahar KARAOĞLAN
Senem KUMOVA METİN
Pages
346–359
Publication Date
2022-10-11
DOI
—
🏛️Bibliographic metadata and full text access for this article are provided via DergiPark.
DergiPark Record ↗
✨

AI Summary & Key Takeaways

AI Insight
⚡ TL;DR(Too Long; Didn't Read)

On the other hand, the classification methods applied on English corpora showed no improvement in the performance of stop word detection.

📌 3 Key Takeaways

🎯
Objective & Hypothesis:In a wide group of languages, the stop words, which have only grammatical roles and not contributing to information content, may be simply exposed by their relatively higher occurrence frequencies.
📊
Key Findings & Data:The evaluations demonstrated that in Turkish corpus, the classification methods improve stop word detection with respect to frequency-based method.
💡
Conclusion & Impact:On the other hand, the classification methods applied on English corpora showed no improvement in the performance of stop word detection.
🔬Design:Research Article
📍Scope:Comprehensive Sample
⚙️Variables:Domain Parameters

Abstract

In a wide group of languages, the stop words, which have only grammatical roles and not contributing to information content, may be simply exposed by their relatively higher occurrence frequencies. But, in agglutinative or inflectional languages, a stop word may be observed in several different surface forms due to the inflection producing noise. In this study, some of the well-known binary classification methods are employed to overcome the inflectional noise problem in stop word detection. The experiments are conducted on corpora of an agglutinative language, Turkish, in which the amount of inflection is high and a non-agglutinative language, English, in which the inflection is lower for stop words. The evaluations demonstrated that in Turkish corpus, the classification methods improve stop word detection with respect to frequency-based method. On the other hand, the classification methods applied on English corpora showed no improvement in the performance of stop word detection.

🔖

Cite This Article

KARAOĞLAN, B., METİN, S. K. (2022). STOP WORD DETECTION AS A BINARY CLASSIFICATION PROBLEM. Anadolu Üniversitesi Bilim ve Teknoloji Dergisi :A-Uygulamalı Bilimler ve Mühendislik, Cilt: 18 - Sayı: 2, 346–359. https://ajindex.com/articles/436598

📄 Full Text PDF Viewer

📄

PDF görüntüleyici hazırlanıyor...

References (12)

  1. Herdan G. Type-token mathematics. A textbook of mathematical linguistics. ‘S-Gravenhage, the Netherlands: Mouton, 1960
  2. Dolamic L, Savoy J. When stopword lists make the difference. Journal of the American Society for Information Science and Technology 2009
  3. Bond F, Shirai S. A Hybrid Rule and Example-based Method for Machine Translation. In M. Carl & A. Way (Eds.), Recent Advances in Example-based Translation 2003:211-224
  4. Martínez-Santiago F, García-Cumbreras MA., Díaz-Galiano MC, Ureña LA. Using Machine Translation Resources with a Mixed 2-Step RSV Merging Algorithm. In C. Peters, P. Clough, J. Gonzalo, G. J. F. Jones, M. Kluck, & B. Magnini (Eds.), Multilingual Information Access for Text, Speech and Images (SINAI at CLEF 2004); 2004; pp. 156-164; Heidelberg: Springer Berlin
  5. Jiang X, Fan X, Wang Y,Jia K. Improving the Performance of Text Categorization Using Automatic Summarization. In S. S. Mahmoud, K. Jusoff and K. Li (Eds.), International Conference on Computer Modeling and Simulation (ICCMS); 2009; pp.347-351. Danvers [6] Radev DR, Fan W. Automatic summarization of search engine hit lists. Proceedings of the ACL- 2000 workshop on Recent advances in natural language processing and information retrieval: held in conjunction with the 38th Annual Meeting of the Association for Computational Linguistics; 2004; pp.99-109; Morristown: Association for Computational Linguistics
  6. Witten IH, Moffat A, Bell TC. Managing Gigabytes: Compressing and Indexing Documents and Images. San Francisco: Morgan Kaufmann, 1999
  7. Manning CD, Schütze H. Foundations of Statistical Natural Language Processing. London: The MIT Press, 2000
  8. Lazarinis F. Engineering and utilizing a stopword list in Greek web retrieval. Journal of the American Society for Information Science and Technology 2007;58(11): 1645–1652
  9. Savoy J. A stemming procedure and stopword list for general French corpora. Journal of the American Society for Information Science and Technology 1999; 50(10): 944-952
  10. Can F, Koçberber S, Balçık E, Kaynak C, Öcalan HÇ, Vursavas O. Information Retrieval on Turkish Texts. Journal of the American Society for Information Science and Technology 2008; 59 (3): 407-421
  11. Zou F, Wang FL, Deng X, Han S. Automatic identification of Chinese stop words. Research on Computing Science 2006; 18: 151–162
  12. Tsz-Wai Lo, R., He, B. and Ounis, I. Automatically building a stopword list for an information retrieval system. Journal of Digital In