KaydedilenlerDerginizi ekleyin

Dergiler / Bilgisayar Bilimleri / 2023 / Cilt: IDAP-2023 : International Artificial Intelligence and Data Processing Symposium Sayı: IDAP-2023 Özel Sayı

Tweet Toplama, Analiz ve Depolama için Platform Tasarımı (TweetCASP)

Designing a platform for Tweet Collection, Analytics and Storage (TweetCASP)

Tugba Beril Doguc(INONU UNIVERSITY)0000-0001-7779-816X
Ahmet Arif Aydın(İNÖNÜ ÜNİVERSİTESİ)0000-0002-4124-7275
Sayfa
165–171
Yayın Tarihi
2023-10-18
🏛️Bu makalenin bibliyografik verileri ve tam metin bağlantısı DergiPark üzerinden sağlanmıştır.
DergiPark Kaydı ↗
✨

Yapay Zeka Hızlı Bakış & Çıkarımlar

AI Insight
⚡ TL;DR(1 Cümlede Hızlı Çıkarım)

Ayrıca, TweetCASP, yazılım geliştiriciler, tasarımcılar ve bu alandaki araştırmacılar için veri toplama ve analiz için kullanılan veri yoğunluklu bir sistem örneği sunar.

📌 3 Maddede Bu Makale

🎯
Amaç & Hipotez:Bu büyük veri çağında, büyük miktarda verinin akışı, depolanması ve analizi çeşitli zorluklar ortaya çıkarır.
📊
Temel Bulgular & Veri:Yararlı bilgilere ulaşmak için yoğun veri kullanan sistemlerin tasarımcıları tarafından çeşitli zorluklar ele alınmalıdır.
💡
Sonuç & Katkı:Ayrıca, TweetCASP, yazılım geliştiriciler, tasarımcılar ve bu alandaki araştırmacılar için veri toplama ve analiz için kullanılan veri yoğunluklu bir sistem örneği sunar.
🔬Tür:Araştırma Makalesi
📍Kapsam:Genişletilmiş Akademik Örneklem
⚙️Değişkenler:Büyük veri · veri yoğunluklu sistemler · gerçek zamanlı analiz

Özet

Bu büyük veri çağında, büyük miktarda verinin akışı, depolanması ve analizi çeşitli zorluklar ortaya çıkarır. Yararlı bilgilere ulaşmak için yoğun veri kullanan sistemlerin tasarımcıları tarafından çeşitli zorluklar ele alınmalıdır. Verilerin toplanması, saklanması ve analiz edilmesi, anlamlı öngörüler elde etmek için uygun veri işleme ve analitik teknolojilerinden oluşan bir toplama ve analiz platformu gerektirir. Bu çalışmada, Twitter'ın Streaming API'sini kullanarak kullanıcı tarafından girilen anahtar kelimelere dayalı olarak tweet'leri toplayan, akış verileri üzerinde gerçek zamanlı analitik için bir ortam sağlayan ve gelecekteki toplu iş odaklı görevleri yerine getirmek için verileri bir Apache Cassandra NoSQL veri deposunda kalıcı olarak depolayan TweetCASP (Tweet toplama, analiz ve depolama platformu) sistemini sunuyoruz. Ayrıca, TweetCASP, yazılım geliştiriciler, tasarımcılar ve bu alandaki araştırmacılar için veri toplama ve analiz için kullanılan veri yoğunluklu bir sistem örneği sunar.

Abstract

In this era of big data, the streaming, storage, and analysis of large amounts of data present a variety of challenges. Several challenges must be addressed by designers of data-intensive systems in order to retrieve useful information. Collecting, storing, and analyzing data requires a collection and analytics platform comprised of an appropriate choice of data processing and analytics technologies in order to acquire meaningful insight. In this paper, we report on TweetCASP (Tweet Collection, Analytics and Storage Platfrom), which gathers tweets based on user-entered keywords using Twitter's Streaming API, providing an environment for real-time analytics on streaming data and permanently storing data in an Apache Cassandra NoSQL datastore to fulfill future batch-oriented data processing requirements. Moreover, The TweetCASP presents an example of a data-intensive system used by software developers, designers, and researchers for data collecting and analytics.

🔖

Bu Makaleye Atıf Yap

Doguc, T. B., Aydın, A. A. (2023). Tweet Toplama, Analiz ve Depolama için Platform Tasarımı (TweetCASP). Bilgisayar Bilimleri, Cilt: IDAP-2023 : International Artificial Intelligence and Data Processing Symposium Sayı: IDAP-2023 Özel Sayı, 165–171. https://doi.org/10.53070/bbd.1344271

📄 Tam Metin PDF Okuyucu

📄

PDF görüntüleyici hazırlanıyor...

Kaynakça (29)

  1. Amghar, S., Cherdal, S., & Mouline, S. (2020). Storing, preprocessing and analyzing tweets: finding the suitable noSQL system. https://doi.org/10.1080/1206212X.2020.1846946
  2. Anderson, K. M., Aydin, A. A., Barrenechea, M., Cardenas, A., Hakeem, M., & Jambi, S. (2015). Design Challenges/Solutions for Environments Supporting the Analysis of Social Media Data in Crisis Informatics Research. 2015 48th Hawaii International Conference on System Sciences, 2015-March, 163–172. https://doi.org/10.1109/HICSS.2015.29
  3. Anderson, K. M., & Schram, A. (2011). Design and implementation of a data analytics infrastructure in support of crisis informatics research: NIER track. 2011 33rd International Conference on Software Engineering (ICSE), 844–847. https://doi.org/10.1145/1985793.1985920
  4. ApacheCassandra. (2022). ApacheCassandra.pdf. https://cassandra.apache.org/_/index.html Aswathy, A., Prabha, R., Gopal, L. S., Pullarkatt, D., & Ramesh, M. V. (2022). An efficient twitter data collection and analytics framework for effective disaster management. 2022 IEEE Delhi Section Conference, DELCON 2022. https://doi.org/10.1109/DELCON54057.2022.9753627
  5. Aydin, A. A. (2016). INCREMENTAL DATA COLLECTION & ANALYTICS THE DESIGN OF NEXT-GENERATION CRISIS INFORMATICS SOFTWARE [Ph.D., University of Colorado Boulder].
  6. https://www.proquest.com/pagepdf/1834583278/Record/9F7C2D640FDE4BCCPQ/3?accountid=16268 Aydin, A. A., & Anderson, K. M. (2017). Batch to Real-Time: Incremental Data Collection & Analytics Platform.
  7. Proceedings of the 50th Hawaii International Conference on System Sciences, 5911–5920. http://hdl.handle.net/10125/41876
  8. Aydin, A. A., & Anderson, K. M. (2020). Data modelling for large-scale social media analytics: design challenges and lessons learned. International Journal of Data Mining, Modelling and Management, 12(4), 386. https://doi.org/10.1504/IJDMMM.2020.111409
  9. Brewer, E. (2012). CAP Twelve Years Later: How the “Rules” Have Changed. Computer, February. https://doi.org/10.1109/MC.2012.37
  10. DB-ENGINES. (2022). DB-Engines Ranking. https://db-engines.com/en/ranking Doguc, T. B., & Aydin, A. A. (2019). CAP-based Examination of Popular NoSQL Database Technologies in Streaming Data Processing. 2019 International Artificial Intelligence and Data Processing Symposium (IDAP), 1–6. https://doi.org/10.1109/IDAP.2019.8875874
  11. Domo Company. (2023). Data Never Sleeps 9.0. https://www.domo.com/learn/infographic/data-never-sleeps-9 Flask Documentation (2.3.x). (2023). https://flask.palletsprojects.com/en/2.3.x/tutorial/database/ Gartner Inc. (2022). Gartner. https://www.gartner.com/en/glossary/all-terms
  12. Gehring, M., Charfuelan, M., & Markl, V. (2019). A comparison of distributed stream processing systems for time series analysis. Lecture Notes in Informatics (LNI), Proceedings - Series of the Gesellschaft Fur Informatik (GI), P-290, 205–214. https://doi.org/10.18420/btw2019-ws-21
  13. Han, H., Yonggang, W., Tat-Seng, C., & Xuelong, L. (2014). Toward Scalable Systems for Big Data Analytics: A Technology Tutorial. Access, IEEE, 2, 652–687. https://doi.org/0.11 09/ACCESS.2014.2332453
  14. Jambi, S., & Anderson, K. M. (2017). Engineering scalable distributed services for real-time big data analytics. Proceedings - 3rd IEEE International Conference on Big Data Computing Service and Applications, BigDataService 2017, 131–140. https://doi.org/10.1109/BigDataService.2017.22
  15. KEKEVİ, U., & AYDIN, A. A. (2022). Real-Time Big Data Processing and Analytics: Concepts, Technologies, and Domains. Computer Science, 55(35), 1–100. https://doi.org/10.53070/bbd.1204112
  16. Laksham Avinash, & Prashant Malik. (2010). Cassandra: a decentralized structured storage system. ACM SIGOPS Operating Systems Review, 1–6. https://doi.org/10.1145/1773912.1773922
  17. Lv, Z., Song, H., Basanta-Val, P., Steed, A., & Jo, M. (2017). Next - Generation Big Data Analytics: State of the. IEEE Transactions on Industrial Informatics, 3203(4), 1891–1899.
  18. Marungo, F. (2018). A primer on NoSQL databases for enterprise architects: The cap theorem and transparent data access with MongoDB and Cassandra. Proceedings of the Annual Hawaii International Conference on System Sciences, 2018-Janua, 4621–4630. https://doi.org/10.24251/hicss.2018.583
  19. RabbitMQ. (2023). https://www.rabbitmq.com/
  20. Roesslein, J. (2022). Tweepy. https://www.tweepy.org/
  21. Rostanski, M., Grochla, K., & Seman, A. (2014). Evaluation of highly available and fault-tolerant middleware clustered architectures using RabbitMQ. 2014 Federated Conference on Computer Science and Information Systems, FedCSIS 2014, 879–884. https://doi.org/10.15439/2014F48
  22. Satauri, I., Satouri, B., & El Beqqali, O. (2023). Big Data Analysis in Commercial Social Networks: Analysis of Twitter Reviews for Marketing Decision Making. European Journal of Information Technologies and Computer Science, 3(2), 1–6. https://doi.org/10.24018/compute.2023.3.2.94
  23. Syed, D., Zainab, A., Ghrayeb, A., Refaat, S. S., Abu-Rub, H., & Bouhali, O. (2021). Smart Grid Big Data Analytics: Survey of Technologies, Techniques, and Applications. IEEE Access, 9, 59564–59585. https://doi.org/10.1109/ACCESS.2020.3041178
  24. Twitter. (2022). Twitter Streaming API. https://developer.twitter.com/en/docs/twitter-api/v1/tweets/filter-realtime/overview
  25. Vanam, H., & R, J. R. R. (2023). Sentiment Analysis of Twitter Data Using Big Data Analytics and Deep Learning Model. 2023 International Conference on Artificial Intelligence and Knowledge Discovery in Concurrent Engineering (ICECONF), 1–6. https://doi.org/10.1109/ICECONF57129.2023.10084281
  26. Yaqoob, I., Hashem, I. A. T., Gani, A., Mokhtar, S., Ahmed, E., Anuar, N. B., & Vasilakos, A. v. (2016). Big data: From beginning to future. International Journal of Information Management, 36(6), 1231–1247. https://doi.org/10.1016/j.ijinfomgt.2016.07.009
  27. Amghar, S., Cherdal, S., & Mouline, S. (2020). Storing, preprocessing and analyzing tweets: finding the suitable noSQL system. https://doi.org/10.1080/1206212X.2020.1846946
  28. https://www.proquest.com/pagepdf/1834583278/Record/9F7C2D640FDE4BCCPQ/3?accountid=16268 Aydin, A. A., & Anderson, K. M. (2017). Batch to Real-Time: Incremental Data Collection & Analytics Platform.
  29. Lv, Z., Song, H., Basanta-Val, P., Steed, A., & Jo, M. (2017). Next - Generation Big Data Analytics: State of the. IEEE Transactions on Industrial Informatics, 3203(4), 1891–1899.