Journals / Konya Journal of Engineering Sciences / 2020 / Cilt: 8 Sayı: 4

Disease Detection From Twitter Data Using Natural Language Processing and Machine Learning

TWİTTER VERİLERİNDEN DOĞAL DİL İŞLEME VE MAKİNE ÖĞRENMESİ İLE HASTALIK TESPİTİ

Pages
839–852
Publication Date
2020-12-01
DOI
—
🏛️Bibliographic metadata and full text access for this article are provided via DergiPark.
DergiPark Record ↗
✨

AI Summary & Key Takeaways

AI Insight
⚡ TL;DR(Too Long; Didn't Read)

The clustering algorithms which are unsupervised methods achieved an accuracy level of %68.60, while the performance of the supervised classification algorithms reached to the accuracy level of %97.48.

📌 3 Key Takeaways

🎯
Objective & Hypothesis:In this study, we determined whether the subject of the messages of the twitter users were about a disease and what kind of diseases they were.
📊
Key Findings & Data:For this purpose, supervised and unsupervised machine learning algorithms were tested and compared using the features extracted via TF-IDF and BOW methods.
💡
Conclusion & Impact:The clustering algorithms which are unsupervised methods achieved an accuracy level of %68.60, while the performance of the supervised classification algorithms reached to the accuracy level of %97.48.
🔬Design:Modeling & Simulation
📍Scope:Comprehensive Sample
⚙️Variables:Twitter · Hastalık Tanıma · Doğal Dil İşleme

Abstract

In this study, we determined whether the subject of the messages of the twitter users were about a disease and what kind of diseases they were. For this purpose, supervised and unsupervised machine learning algorithms were tested and compared using the features extracted via TF-IDF and BOW methods. Data were collected with Python scripts from Twitter. The Scikit-Learn library which was developed for Python was used to implement the algorithms. The clustering algorithms which are unsupervised methods achieved an accuracy level of %68.60, while the performance of the supervised classification algorithms reached to the accuracy level of %97.48.

Özet

Bu çalışmada twitterdaki kullanıcıların yazmış oldukları mesajların hastalık konulu olup olmadığı ve hastalık türleri tespit edilmiştir. Bu amaçla gözetimli ve gözetimsiz makine öğrenmesi algoritmaları, TF-IDF ve BOW yöntemleri ile çıkarılan özellikler ile denenmiş ve karşılaştırmalar yapılmıştır. Veriler Python betikleri ile twitter üzerinden toplanmıştır. Algoritmaları uygulamak için Python için geliştirilmiş Scikit-Learn kütüphanesi kullanılmıştır. Gözetimsiz olarak verilerin kümelenmesinde %68.60’lık bir başarı elde edilirken, gözetimli algoritmalar ile yapılan sınıflandırmalarda %97.48’lik başarı oranına ulaşılmıştır.

🔖

Cite This Article

ÖZTÜRK, A., DURAK, Ü., BADILLI, F. (2020). Disease Detection From Twitter Data Using Natural Language Processing and Machine Learning. Konya Journal of Engineering Sciences, Cilt: 8 Sayı: 4, 839–852. https://ajindex.com/articles/909132

📄 Full Text PDF Viewer

📄

PDF görüntüleyici hazırlanıyor...

References (12)

  1. Aloise, D., Deshpande, A., Hansen, P., Popat, P., 2009, "NP-hardness of Euclidean sum-of-square clustering", Machine learning, Cilt 75, Sayı 2, ss. 245-248.
  2. Ambert, K. H., Cohen, A.M., 2009, “A System for Classifying Disease Comorbidity Status from Medical Discharge Summaries Using Automated Hotspot and Negated Concept Detection”, Journal of the American Medical Informatics Association, Cilt 16, Sayı 4, ss. 590–595.
  3. Acherkar, H., Gandhe, A., Lazarus, R., Yu, S., Liu, B., 2011, “Predicting Flu Trends using Twitter Data”, 2011 IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS), Shanghai, China, 702-706.
  4. Cavnar, W. B., Trenkle, J. M., 1994, "N-gram-based text categorization.", Proceedings of SDAIR-94, 3rd annual symposium on document analysis and information retrieval, Las Vegas, Nevada, A.B.D., 161- 175. Conmay, M., Hu, M., Chapman W.W., 2019, “Recent Advances in Using Natural Language Processing to Address Public Health Research Questions Using Social Media and ConsumerGenerated Data”, Yearbook of Medical Informatics, Cilt 28, Sayı 1, ss. 208-217.
  5. Dai, X., Bikdash, M., 2015, “Hybrid Classification for Tweets Related to Infection with Influenza”, Proceedings of the IEEE SoutheastCon 2015,
  6. Fort Lauderdale, Florida, 1-5.
  7. Dai, X., Bikdash, M., 2016, “Distance-based Outliers Method for Detecting Disease Outbreaks using Social Media”, Proceedings of the IEEE SoutheastCon 2015,
  8. Norfolk, VA, USA, 1-8.
  9. Edo-Osagie, O., Iglesia, B.D.L., Lake, I., Edeghere, O., 2020, “A scoping review of the use of Twitter for public health research”, Computers in Biology and Medicine, Available Online, 103770, doi: 10.1016/j.compbiomed.2020.103770. Hartigan, J.A., Wong, M. A., 1979, "Algorithm AS 136: A k-means clustering algorithm", Journal of the Royal Statistical Society, Series C (Applied Statistics), Cilt 28, Sayı 1, ss. 100-108.
  10. Kohavi, R., 1995, "A study of cross-validation and bootstrap for accuracy estimation and model selection", IJCAI'95 Proceedings of The 14th International Joint Conference on Artificial Intelligence, Montreal, Quebec, Canada, 2: 1137-1143.
  11. Lerman, P.M., 1980, "Fitting segmented regression models by grid search", Journal of the Royal Statistical Society: Series C (Applied Statistics), Cilt 29, Sayı 1, ss. 77-84.
  12. Manning, C., Schütze, H., 1999, “Foundations of Statistical Natural Language Processing”, MIT press, Cambridge, MA, A.B.D. Morita, M., Maskawa, Aramaki, S., E., 2013, “Comparing Social Media and Search Activity as Social Sensors