SavedAdd your journal

Journals / Dokuz Eylül Üniversitesi Mühendislik Fakültesi Fen ve Mühendislik Dergisi / 2026 / Cilt: 24 Sayı: 71

Classification of Scatter Plot Images Using Deep Learning

Derin Öğrenme Kullanarak Dağılım Grafiği Görüntülerinin Sınıflandırılması

Derya Bırant(DOKUZ EYLUL UNIVERSITY)0000-0003-3138-0432
Aydanur Akça(DOKUZ EYLUL UNIVERSITY)0000-0002-9146-0613
Buse Bozkurt(DOKUZ EYLUL UNIVERSITY)0000-0001-6086-8109
Mehtap Bağlan(DOKUZ EYLUL UNIVERSITY)0000-0001-8813-9469
Pages
631–642
Publication Date
2022-05-16
🏛️Bibliographic metadata and full text access for this article are provided via DergiPark.
DergiPark Record ↗
✨

AI Summary & Key Takeaways

AI Insight
⚡ TL;DR(Too Long; Didn't Read)

The experimental results showed that the proposed system successfully (93.90%) classified scatter plot images to help visually impaired users understand the information given in the graph.

📌 3 Key Takeaways

🎯
Objective & Hypothesis:Scatter plot is one of the well-known charts and is frequently embedded in different types of documents such as articles, books, and dissertations.
📊
Key Findings & Data:However, the information given in the scatter plots can’t be directly noticed by visually impaired individuals, because they are usually in an image format, and so they are not naturally readable by machines.
💡
Conclusion & Impact:The experimental results showed that the proposed system successfully (93.90%) classified scatter plot images to help visually impaired users understand the information given in the graph.
🔬Design:Araştırma Makalesi
📍Scope:Comprehensive Sample
⚙️Variables:Dağılım grafikleri · Görüntü sınıflandırma · Derin öğrenme

Abstract

Scatter plot is one of the well-known charts and is frequently embedded in different types of documents such as articles, books, and dissertations. However, the information given in the scatter plots can’t be directly noticed by visually impaired individuals, because they are usually in an image format, and so they are not naturally readable by machines. To solve this problem, this paper proposes a system that can extract visual properties from scatter plot images using deep learning and image processing techniques. It is the first study that automatically classifies scatter plots in terms of two aspects: degree of correlation (strong or weak) and types of correlation (positive, negative, or neutral). In the experimental studies, alternative convolutional neural network (CNN) architectures were compared on both synthetic and real-world datasets in terms of accuracy, including Residual Networks (ResNet), Alex Networks (AlexNet), and Visual Geometry Group (VGG) Networks. The experimental results showed that the proposed system successfully (93.90%) classified scatter plot images to help visually impaired users understand the information given in the graph.

Özet

Dağılım grafiği, iyi bilinen grafiklerden biridir ve makaleler, kitaplar, raporlar gibi birçok farklı türdeki dokümanlarda sıklıkla kullanılmaktadır. Ancak, dağılım grafikleri genellikle görüntü biçiminde olduğu için grafiklerde verilen bilgiler görme engelli kişiler tarafından fark edilemez, yani esasen makine tarafından okunabilir değillerdir. Bu sorunu çözmek için, bu makale, derin öğrenme ve görüntü işleme tekniklerini kullanarak, dağılım grafiği görüntülerinden görsel özellikleri çıkartabilen bir sistem önermektedir. Dağılım grafiklerini iki açıdan otomatik olarak sınıflandıran ilk çalışmadır: korelasyon derecesi (güçlü veya zayıf) ve korelasyon türleri (pozitif, negatif veya nötr). Deneysel çalışmalarda, Artık Ağlar (ResNet), Alex Ağları (AlexNet) ve Görsel Geometri Grubu (VGG) Ağları gibi alternatif evrişimsel sinir ağı (CNN) mimarileri hem sentetik hem de gerçek dünya veri setlerinde doğruluk açısından karşılaştırılmıştır. Deneysel sonuçlar, önerilen sistemin başarılı bir şekilde (%93,90) dağılım grafiği görüntülerini sınıflandırarak görme engelli kullanıcıların grafikte verilen bilgileri anlamalarına yardımcı olduğunu göstermiştir.

🔖

Cite This Article

Bırant, D., Akça, A., Bozkurt, B., Bağlan, M. (2022). Classification of Scatter Plot Images Using Deep Learning. Dokuz Eylül Üniversitesi Mühendislik Fakültesi Fen ve Mühendislik Dergisi, Cilt: 24 Sayı: 71, 631–642. https://doi.org/10.21205/deufmd.2022247126

📄 Full Text PDF Viewer

📄

PDF görüntüleyici hazırlanıyor...

References (46)

  1. Shao, L., Mahajan, A., Schreck, T., Lehmann, D.J. 2017. Interactive Regression Lens for Exploring Scatter Plots, Computer Graphics Forum, Volume. 36, p. 157-166. DOI: 10.1111/cgf.13176
  2. Wang, W.B., Huang, M.L., Nguyen, Q.V., Huang, W., Zhang, K., Huang, T.H. 2016. Enabling Decision Trend Analysis with Interactive Scatter Plot Matrices Visualization, Journal of Visual Languages & Computing, Volume. 33, p. 13-23. DOI: 10.1016/j.jvlc.2015.11.002
  3. Sainani, K.L. 2016. The Value of Scatter Plots, Physical Medicine and Rehabilitation (PM&R), Volume. 8, p. 1213-1217. DOI: 10.1016/j.pmrj.2016.10.018
  4. Mohseni, F., Mokhtarzade, M. 2021. The Synergistic Use of Microwave Coarse-scale Measurements and Two Adopted High-resolution Indices Driven from Long-term T-V Scatter Plot for Fine-scale Soil Moisture Estimation, GIScience & Remote Sensing, Volume. 58, p. 455-482. DOI: 10.1080/15481603.2021.1906056
  5. Zhang, Z., Cui, X., Jeske, D.R., Borneman, J. 2013. Biclustering Scatter Plots Using Data Depth Measures, Statistical Analysis and Data Mining, Volume. 6, p. 102-115. DOI: 10.1002/sam.11166
  6. Fu, J., Zhu, B., Cui, W., Ge, S., Wang, Y., Zhang, H., Huang, H., Tang, Y., Zhang, D., Ma, X. 2020. Chartem: Reviving Chart Images with Data Embedding, IEEE Transactions on Visualization and Computer Graphics, Volume. 27, p. 337-346. DOI: 10.1109/TVCG.2020.3030351
  7. Al-Zaidy, R.A., Giles, C.L. 2015. Automatic Extraction of Data from Bar Charts. 8th International Conference on Knowledge Capture, 07-10 October, Palisades, USA, 1-4. DOI: 10.1145/2815833.2816956
  8. Bajic, F., Job, J., Nenadic, K. 2019. Chart Classification Using Simplified VGG Model, International Conference on Systems, Signals and Image Processing, 5–7 June, Osijek, Croatia, 229-233. DOI: 10.1109/IWSSIP.2019.8787299
  9. Araujo, T., Chagas, P., Alves, J., Santos, C., Santos, B., Meiguins, B. 2020. A Real-World Approach on the Problem of Chart Recognition Using Classification, Detection and Perspective Correction, Sensors, Volume. 20, p. 1-21. DOI: 10.3390/s20164370
  10. Deepa, R., Tamilselvan, E. 2020. Processing Of Evaluation Chart Uses Optical Character Recognition, International Journal of Scientific & Technology Research, Volume. 9, p. 4770-4773.
  11. Vassilieva, N., Fomina, Y. 2013. Text Detection in Chart Images, Pattern Recognition and Image Analysis, Volume. 23, p. 139-144. DOI: 10.1134/S1054661813010112
  12. Chagas, P., Freitas, A., Daisuke, R., Miranda, B., De Araújo, T.D.O., Santos, C., Meiguins, B., De Morais, J.M. 2017. Architecture Proposal for Data Extraction of Chart Images Using Convolutional Neural Network, 21st International Conference on Information Visualisation, 11–14 July, London, UK, 318-323. DOI: 10.1109/iV.2017.37
  13. Zan, T., Liu, Z., Wang, H., Wang, M., Gao, X. 2020. Control Chart Pattern Recognition Using the Convolutional Neural Network, Journal of Intelligent Manufacturing, Volume. 31, p. 703-716. DOI: 10.1007/s10845-019-01473-0
  14. Mishra, P., Kumar, S., Chaube, M.K. 2020. Dissimilarity Based Regularized Deep Learning Model for Information Charts, 9th International Conference on Informatics, Electronics & Vision, 26-29 August, Kitakyushu, Japan, 1-6. DOI: 10.1109/ICIEVicIVPR48672.2020.9306660
  15. Zhou, F., Zhao, Y., Chen, W., Tan, Y., Xu, Y., Chen, Y., Liu, C., Zhao, Y. 2021. Reverse-engineering Bar Charts Using Neural Networks. Journal of Visualization, Volume. 24, p. 419-435. DOI: 10.1007/s12650-020-00702-6
  16. Sohn, C., Choi, H., Kim, K., Park, J., Noh, J. 2021. Line Chart Understanding with Convolutional Neural Network, Electronics, Volume. 10, p. 1-17. DOI: 10.3390/electronics10060749
  17. De, P. 2018. Automatic Data Extraction from 2D and 3D Pie Chart Images, IEEE 8th International Advance Computing Conference, 14–15 December, India, 20-25. DOI: 10.1109/IADCC.2018.8692104
  18. Fuqua, D., Razzaghi, T. 2020. A Cost-sensitive Convolution Neural Network Learning for Control Chart Pattern Recognition, Expert Systems with Applications, Volume. 150, p. 1-17. DOI: 10.1016/j.eswa.2020.113275
  19. Zan, T., Su, Z., Liu, Z., Chen, D., Wang, M., Gao, X. 2020. Pattern Recognition of Different Window Size Control Charts Based on Convolutional Neural Network and Information Fusion, Symmetry, Volume. 12, p. 1-13. DOI: 10.3390/sym13010110
  20. Birogul, S., Temür, G., Kose, U. 2020. YOLO Object Recognition Algorithm and “Buy-Sell Decision” Model Over 2D Candlestick Charts, IEEE Access, Volume. 8, p. 91894-91915. DOI: 10.1109/ACCESS.2020.2994282
  21. Siper, M., Makinen, K., Kanan, R. 2021. TABot – A Distributed Deep Learning Framework for Classifying Price Chart Images, Advanced Computing, Volume. 1367. p. 1-15. DOI: 10.1007/978-981-16-0401-0_37
  22. Ünlü, R. 2021. A Robust Data Simulation Technique to Improve Early Detection Performance of a Classifier in Control Chart Pattern Recognition Systems, Information Sciences, Volume. 548, p. 18-36. DOI: 10.1016/j.ins.2020.09.059
  23. Kaur, P., Kiesel, D. 2020. Combining Image and Caption Analysis for Classifying Charts in Biodiversity Texts, 15th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications, 27-29 February, Valletta, Malta, 157-168. DOI: 10.5220/0008946701570168
  24. Zaman, M., Hassan, A. 2021. Fuzzy Heuristics and Decision Tree for Classification of Statistical Feature-Based Control Chart Patterns, Symmetry, Volume. 13, p. 1-12. DOI: 10.3390/sym13010110
  25. Mishra, P. & Kumar, S., Chaube, M.K. 2020. Interpretation and Segmentation of Chart Images Using h-Means Image Clustering Algorithm, International Conference on Data Management, Analytics and Innovation, 17–19 January, New Delhi, India, 379-391. DOI: 10.1007/978-981-15-5616-6_27
  26. Sharma, M., Gupta, S., Chowdhury, A., Vig, L. 2019. ChartNet: Visual Reasoning Over Statistical Charts Using MAC-Networks, International Joint Conference on Neural Networks, 14-19 July, Budapest, Hungary, 1-7. DOI: 10.1109/IJCNN.2019.8852427
  27. Mishchenko, A., Vassilieva, N. 2011. Chart Image Understanding and Numerical Data Extraction, 6th International Conference on Digital Information Management, 26–28 September, Australia, 115-210. DOI: 10.1109/ICDIM.2011.6093320
  28. Mishchenko, A., Vassilieva, N. 2011. Model-based Chart Image Classification, International Symposium on Visual Computing, Advances in Visual Computing, Lecture Notes in Computer Science, Volume. 6939, p. 476-485. DOI: 10.1007/978-3-642-24031 -7_48
  29. Maboudou-Tchao, E.M. 2020. Change Detection Using Least Squares One-class Classification Control Chart, Quality Technology & Quantitative Management, Volume. 17, p. 609-626. DOI: 10.1080/16843703.2019.1711302
  30. Mishra, P., Kumar, S., Chaube, M.K. 2020. ChartFuse: A Novel Fusion Method for Chart Classification Using Heterogeneous Microstructures, Multimedia Tools and Applications, Volume. 80, p. 10417-10439. DOI: 10.1007/s11042-020-10186-z
  31. Direncioğlu Diren, D., Boran, S., Cil, I. 2020. Integration of Machine Learning Techniques and Control Charts in Multivariate Processes, Scientia Iranica, Volume. 27, p. 3233-3241, DOI: 10.24200/sci.2019.50377.1667
  32. Kalteh, A.A., Babouei, S. 2020. Control Chart Patterns Recognition Using ANFIS with New Training Algorithm and Intelligent Utilization of Shape and Statistical Features, ISA Transactions, Volume. 102, p. 12-22. DOI: 10.1016/j.isatra.2019.12.001
  33. Kadakadiyavar, S., Ramrao, N., Singh, M.K. 2019. Efficient Mixture Control Chart Pattern Recognition Using Adaptive RBF Neural Network, International Journal of Information Technology, Volume. 12, p. 1271-1280. DOI: 10.1007/s41870-019-00381-z
  34. Shao, Y.E., Hu, Y.T. 2020. Using Machine Learning Classifiers to Recognize the Mixture Control Chart Patterns for a Multiple-input Multiple-output Process, Mathematics, Volume. 8, p. 1-14. DOI: 10.3390/math8010102
  35. Dai, W., Wang, M., Niu, Z., Zhang, J. 2018. Chart Decoder: Generating Textual and Numeric Information from Chart Images Automatically, Journal of Visual Languages & Computing, Volume. 48, p. 101-109. DOI: 10.1016/j.jvlc.2018.08.005
  36. Tang, B., Liu, X., Lei, J., Song, M., Tao, D., Sun, S., Dong, F. 2016. DeepChart: Combining Deep Convolutional Networks and Deep Belief Networks in Chart Classification, Signal Process, Volume. 124, p. 156-161. DOI: 10.1016/j.sigpro.2015.09.027
  37. Singh, M., Goyal, P. 2021. ChartSight: An Automated Scheme for Assisting Visually Impaired in Understanding Scientific Charts, 16th International Joint Conference on Computer Vision, 8-10 February, Setubal, Portugal, 309-318. DOI: 10.5220/0010201203090318
  38. Savva, M., Kong, N., Chhajta, A., Li, F.F., Agrawala, M., Heer, J. 2011. ReVision: Automated Classification, Analysis and Redesign of Chart Images, 24th Annual ACM Symposium on User Interface Software and Technology, 16-19 October, California, USA, 393-402. DOI: 10.1145/20471 96.20472 47
  39. Jung, D., Kim, W., Song, H., Hwang, J., Lee, B., Kim, B., Seo, J. 2017. ChartSense: Interactive Data Extraction from Chart Images, Conference on Human Factors in Computing Systems, 6-11 May, Denver Colorado, USA, 6706-6717. DOI: 10.1145/30254 53.30259 57
  40. Krizhevsky, A., Sutskever, I., Hinton, G.E. 2012. ImageNet Classification with Deep Convolutional Neural Networks, International Conference on Neural Information Processing Systems, 3-6 December, Nevada, USA, 1097–1105.
  41. Simonyan, K., Zisserman, A. 2015. Very Deep Convolutional Networks for Large-scale Image Recognition, 3rd International Conference on Learning Representations, 7-9 May, San Diego, CA, USA, 1–14.
  42. He, K., Zhang, X., Ren, S., Sun, J. 2016. Deep Residual Learning for Image Recognition, IEEE Conference on Computer Vision and Pattern Recognition, 26 June-1 July, Las Vegas, USA, 770-778. DOI: 10.1109/CVPR.2016.90
  43. Jia, S., Wang, P., Jia, P., Hu, S. 2017. Research on Data Augmentation for Image Classification Based on Convolution Neural Networks. Chinese Automation Congress, 20-22 October, Jinan, China, 4165-4170. DOI: 10.1109/CAC.2017.8243510
  44. Wicaksono P. 2016. Improving the Accuracy of Multispectral-based Benthic Habitats Mapping Using Image Rotations: The Application of Principle Component Analysis and Independent Component Analysis, European Journal of Remote Sensing, Volume. 49, p. 433-463. DOI: 10.5721/EuJRS20164924
  45. Liang, G., Hong, H., Xie, W., Zheng L. 2018. Combining Convolutional Neural Network With Recursive Neural Network for Blood Cell Image Classification, IEEE Access, Volume. 6, p. 36188-36197. DOI: 10.1109/ACCESS.2018.2846685
  46. Somasundaram, D. 2019. Machine Learning Approach for Homolog Chromosome Classification, International Journal of Imaging Systems and Technology, Volume. 29, p. 161-167. DOI: 10.1002/ima.22287