SavedAdd your journal

Journals / Bilgisayar Bilimleri / 2023 / Cilt: IDAP-2023 : International Artificial Intelligence and Data Processing Symposium Sayı: IDAP-2023

Designing a platform for Tweet Collection, Analytics and Storage (TweetCASP)

Tweet Toplama, Analiz ve Depolama için Platform Tasarımı (TweetCASP)

Tugba Beril DOGUC
Ahmet Arif AYDIN
Pages
165–171
Publication Date
2023-10-18
DOI
—
🏛️Bibliographic metadata and full text access for this article are provided via DergiPark.
DergiPark Record ↗
✨

AI Summary & Key Takeaways

AI Insight
⚡ TL;DR(Too Long; Didn't Read)

Moreover, The TweetCASP presents an example of a data-intensive system used by software developers, designers, and researchers for data collecting and analytics.

📌 3 Key Takeaways

🎯
Objective & Hypothesis:In this era of big data, the streaming, storage, and analysis of large amounts of data present a variety of challenges.
📊
Key Findings & Data:Several challenges must be addressed by designers of data-intensive systems in order to retrieve useful information.
💡
Conclusion & Impact:Moreover, The TweetCASP presents an example of a data-intensive system used by software developers, designers, and researchers for data collecting and analytics.
🔬Design:Research Article
📍Scope:Comprehensive Sample
⚙️Variables:Büyük veri · veri yoğunluklu sistemler · gerçek zamanlı analiz

Abstract

In this era of big data, the streaming, storage, and analysis of large amounts of data present a variety of challenges. Several challenges must be addressed by designers of data-intensive systems in order to retrieve useful information. Collecting, storing, and analyzing data requires a collection and analytics platform comprised of an appropriate choice of data processing and analytics technologies in order to acquire meaningful insight. In this paper, we report on TweetCASP (Tweet Collection, Analytics and Storage Platfrom), which gathers tweets based on user-entered keywords using Twitter's Streaming API, providing an environment for real-time analytics on streaming data and permanently storing data in an Apache Cassandra NoSQL datastore to fulfill future batch-oriented data processing requirements. Moreover, The TweetCASP presents an example of a data-intensive system used by software developers, designers, and researchers for data collecting and analytics.

Özet

Bu büyük veri çağında, büyük miktarda verinin akışı, depolanması ve analizi çeşitli zorluklar ortaya çıkarır. Yararlı bilgilere ulaşmak için yoğun veri kullanan sistemlerin tasarımcıları tarafından çeşitli zorluklar ele alınmalıdır. Verilerin toplanması, saklanması ve analiz edilmesi, anlamlı öngörüler elde etmek için uygun veri işleme ve analitik teknolojilerinden oluşan bir toplama ve analiz platformu gerektirir. Bu çalışmada, Twitter'ın Streaming API'sini kullanarak kullanıcı tarafından girilen anahtar kelimelere dayalı olarak tweet'leri toplayan, akış verileri üzerinde gerçek zamanlı analitik için bir ortam sağlayan ve gelecekteki toplu iş odaklı görevleri yerine getirmek için verileri bir Apache Cassandra NoSQL veri deposunda kalıcı olarak depolayan TweetCASP (Tweet toplama, analiz ve depolama platformu) sistemini sunuyoruz. Ayrıca, TweetCASP, yazılım geliştiriciler, tasarımcılar ve bu alandaki araştırmacılar için veri toplama ve analiz için kullanılan veri yoğunluklu bir sistem örneği sunar.

🔖

Cite This Article

DOGUC, T. B., AYDIN, A. A. (2023). Designing a platform for Tweet Collection, Analytics and Storage (TweetCASP). Bilgisayar Bilimleri, Cilt: IDAP-2023 : International Artificial Intelligence and Data Processing Symposium Sayı: IDAP-2023, 165–171. https://ajindex.com/articles/968714

📄 Full Text PDF Viewer

📄

PDF görüntüleyici hazırlanıyor...

References (13)

  1. Amghar, S., Cherdal, S., & Mouline, S. (2020). Storing, preprocessing and analyzing tweets: finding the suitable noSQL system. https://doi.org/10.1080/1206212X.2020.1846946
  2. Anderson, K. M., Aydin, A. A., Barrenechea, M., Cardenas, A., Hakeem, M., & Jambi, S. (2015). Design Challenges/Solutions for Environments Supporting the Analysis of Social Media Data in Crisis Informatics Research. 2015 48th Hawaii International Conference on System Sciences, 2015-March, 163–172. https://doi.org/10.1109/HICSS.2015.29
  3. Anderson, K. M., & Schram, A. (2011). Design and implementation of a data analytics infrastructure in support of crisis informatics research: NIER track. 2011 33rd International Conference on Software Engineering (ICSE), 844–847. https://doi.org/10.1145/1985793.1985920
  4. ApacheCassandra. (2022). ApacheCassandra.pdf. https://cassandra.apache.org/_/index.html
  5. Aswathy, A., Prabha, R., Gopal, L. S., Pullarkatt, D., & Ramesh, M. V. (2022). An efficient twitter data collection and analytics framework for effective disaster management. 2022
  6. IEEE Delhi Section Conference, DELCON 2022. https://doi.org/10.1109/DELCON54057.2022.9753627
  7. Aydin, A. A. (2016). INCREMENTAL DATA COLLECTION & ANALYTICS THE DESIGN OF NEXT-GENERATION CRISIS INFORMATICS SOFTWARE [Ph.D., University of Colorado Boulder]. https://www.proquest.com/pagepdf/1834583278/Record/9F7C2D640FDE4BCCPQ/3?accountid=16268
  8. Aydin, A. A., & Anderson, K. M. (2017). Batch to Real-Time: Incremental Data Collection & Analytics Platform. Proceedings of the 50th Hawaii International Conference on System Sciences, 5911–5920. http://hdl.handle.net/10125/41876
  9. Aydin, A. A., & Anderson, K. M. (2020). Data modelling for large-scale social media analytics: design challenges and lessons learned. International Journal of Data Mining, Modelling and Management, 12(4), 386. https://doi.org/10.1504/IJDMMM.2020.111409
  10. Brewer, E. (2012). CAP Twelve Years Later: How the “Rules” Have Changed. Computer, February. https://doi.org/10.1109/MC.2012.37
  11. DB-ENGINES. (2022). DB-Engines Ranking. https://db-engines.com/en/ranking
  12. Doguc, T. B., & Aydin, A. A. (2019). CAP-based Examination of Popular NoSQL Database Technologies in Streaming Data Processing. 2019 International Artificial Intelligence and Data Processing Symposium (IDAP), 1–6. https://doi.org/10.1109/IDAP.2019.8875874
  13. Domo Company. (2023). Data Never Sleeps 9.0. https://www.domo.com/learn/infographic/data-never-sleeps-9 Flask Documentation (2.3.x). (2023). https://flask.palletsproj