SavedAdd your journal

Journals / Fırat Üniversitesi Fen Bilimleri Dergisi / 2026 / Cilt: 38 Sayı: 2

Comparative Evaluation of CNN Architectures for Pneumonia Detection: The Impact of Validation Set Size and Batch Normalization on Model Performance

Zatürre Tespitinde CNN Mimarilerinin Karşılaştırmalı Değerlendirilmesi: Doğrulama Kümesi Büyüklüğünün ve Batch Normalizasyonun Model Performansı Üzerindeki Etkisi

Rasim Çekik(ŞIRNAK ÜNİVERSİTESİ)0000-0002-7820-413X
Pages
109–124
Publication Date
2026-09-30
🏛️Bibliographic metadata and full text access for this article are provided via DergiPark.
DergiPark Record ↗
✨

AI Summary & Key Takeaways

AI Insight
⚡ TL;DR(Too Long; Didn't Read)

Overall, the findings demonstrate that model performance cannot be explained solely by architectural capacity; validation set adequacy and normalization behavior are fundamental factors influencing both performance and model ranking.

📌 3 Key Takeaways

🎯
Objective & Hypothesis:Convolutional neural networks (CNNs) can achieve high accuracy in pneumonia detection from chest radiographs; however, the interaction between architectural complexity and dataset characteristics remains insufficiently investigated.
📊
Key Findings & Data:On PneumoniaMNIST, DeeperCNN achieved 84.78% accuracy and an F1 score of 0.8366, whereas SimpleCNN achieved the highest performance on the Kaggle dataset, with 88.30% accuracy and an F1 score of 0.9077.
💡
Conclusion & Impact:Overall, the findings demonstrate that model performance cannot be explained solely by architectural capacity; validation set adequacy and normalization behavior are fundamental factors influencing both performance and model ranking.
🔬Design:Modeling & Simulation
📍Scope:Comprehensive Sample
⚙️Variables:evrişimsel sinir ağları · zatürre tespiti · tıbbi görüntü sınıflandırma

Abstract

Convolutional neural networks (CNNs) can achieve high accuracy in pneumonia detection from chest radiographs; however, the interaction between architectural complexity and dataset characteristics remains insufficiently investigated. In this study, SimpleCNN, DeeperCNN, and WideCNN architectures were compared on two datasets differing in resolution, class balance, and validation set size: PneumoniaMNIST and the Kaggle chest X-ray dataset. On PneumoniaMNIST, DeeperCNN achieved 84.78% accuracy and an F1 score of 0.8366, whereas SimpleCNN achieved the highest performance on the Kaggle dataset, with 88.30% accuracy and an F1 score of 0.9077. In contrast, DeeperCNN failed on the latter dataset, achieving 38.46% accuracy, 1.0000 precision, and 0.0154 recall. Although training accuracy reached 92.5%, validation accuracy remained at 50%, indicating a mismatch between training and inference statistics in the batch normalization layers. Upon reevaluation, ROC-AUC remained within the range of 0.77–0.80, while accuracy increased from 62.5% to 71.8%. Furthermore, the 16-image validation set led to model selection that was inconsistent with test performance. Overall, the findings demonstrate that model performance cannot be explained solely by architectural capacity; validation set adequacy and normalization behavior are fundamental factors influencing both performance and model ranking.

Özet

Evrişimsel sinir ağları (CNN), göğüs radyografilerinden pnömoni tespitinde yüksek doğruluk sağlayabilmekte, ancak mimari karmaşıklık ile veri kümesi özellikleri arasındaki etkileşim yeterince incelenmemektedir. Bu çalışmada SimpleCNN, DeeperCNN ve WideCNN mimarileri; çözünürlük, sınıf dengesi ve doğrulama seti büyüklüğü bakımından farklı iki veri kümesi olan PneumoniaMNIST ve Kaggle göğüs röntgeni veri kümesi üzerinde karşılaştırılmıştır. PneumoniaMNIST'te DeeperCNN %84,78 doğruluk ve 0,8366 F1 skoru elde ederken, Kaggle veri kümesinde SimpleCNN %88,30 doğruluk ve 0,9077 F1 skoru ile en yüksek performansı göstermiştir. Buna karşılık DeeperCNN, ikinci veri kümesinde %38,46 doğruluk, 1,0000 kesinlik ve 0,0154 duyarlılık ile başarısız olmuştur. Eğitim doğruluğunun %92,5'e ulaşmasına rağmen doğrulama doğruluğunun %50'de kalması, toplu normalleştirme katmanlarının eğitim ve çıkarım istatistikleri arasındaki uyumsuzluğa işaret etmiştir. Yeniden değerlendirme sonucunda ROC-AUC 0,77–0,80 aralığında korunurken doğruluk %62,5'ten %71,8'e yükselmiştir. Ayrıca 16 görüntülük doğrulama seti, test performansıyla uyumsuz model seçimine yol açmıştır. Sonuç olarak bulgular, model performansının yalnızca mimari kapasiteyle açıklanamayacağını; doğrulama setinin yeterliliği ile normalleştirme davranışının performans ve model sıralamasını belirleyen temel etmenler olduğunu ortaya koymaktadır.

🔖

Cite This Article

Çekik, R. (2026). Comparative Evaluation of CNN Architectures for Pneumonia Detection: The Impact of Validation Set Size and Batch Normalization on Model Performance. Fırat Üniversitesi Fen Bilimleri Dergisi, Cilt: 38 Sayı: 2, 109–124. https://doi.org/10.66605/fufbd.2012906

📄 Full Text PDF Viewer

📄

PDF görüntüleyici hazırlanıyor...

References (36)

  1. Troeger, C. E., Blacker, B. F., Khalil, I. A., Zimsen, S. R., Albertson, S. B., Abate, D.,. & Rai, R. K. Mortality, morbidity, and hospitalisations due to influenza lower respiratory tract infections, 2017: an analysis for the Global Burden of Disease Study 2017. Lancet Respir Med 2019; 7(1): 69-89.
  2. World Health Organization. "Pneumonia." World Health Organization, 2023, https://www.who.int/health-topics/pneumonia.
  3. Cillóniz C, Fally M, Evans S, Dy-Agra G. The pneumonia challenge: time to end the global neglect. Lancet Glob Health 2025; 13(12): e1990-e1991.
  4. Kazi S, Hernstadt H, Abo YN, Graham H, Palmer M., Graham S. The utility of chest x-ray and lung ultrasound in the management of infants and children presenting with severe pneumonia in low- and middle-income countries: a pragmatic scoping review. J Glob Health 2022; 12: 10013.
  5. Htun TP, Sun Y, Chua HL, Pang J. Clinical features for diagnosis of pneumonia among adults in primary care setting: a systematic and meta-review. Sci Rep 2019; 9(1): 7600.
  6. Rajpurkar P, Irvin J, Ball RL, Zhu K, Yang B., Mehta H.,Duan T., Ding D., et al. Deep learning for chest radiograph diagnosis: a retrospective comparison of the CheXNeXt algorithm to practicing radiologists. PLoS Med 2018; 15(11): e1002686.
  7. Chen KC, Yu HR, Chen WS, Lin WC, Lee YC., Chen HH., Jiang JH., Su TY., et al. Diagnosis of common pulmonary diseases in children by X-ray images and deep learning. Sci Rep 2020; 10: 17374.
  8. López-González R, García J, Fos-Guarinos B, García-Castro F, Alberich-Bayarri Á, Soria-Olivas E., Muñoz-Núñez C., Martí-Bonmatí L. Automated chest radiographs triage reading by a deep learning referee network. medRxiv 2021.
  9. LeCun Y, Bengio Y, Hinton G. Deep learning. Nature 2015; 521(7553): 436-444.
  10. Tang YX, Tang YB, Peng Y, Yan K, Bagheri M., Redd BA., Brandon CJ., Lu Z., et al. Automated abnormality classification of chest radiographs using deep convolutional neural networks. NPJ Digit Med 2020; 3: 70.
  11. Li M, Jiang Y, Zhang Y, Haisheng Z. Medical image analysis using deep learning algorithms. Front Public Health 2023; 11: 1273253.
  12. Litjens G, Kooi T, Bejnordi BE, Setio AAA, Ciompi F., Ghafoorian M., Laak JAWM, Ginnekenet, B., et al. A survey on deep learning in medical image analysis. Med Image Anal 2017; 42: 60-88.
  13. Rajpurkar P, Irvin J, Zhu K, Yang B, Mehta H., Duan T., Ding D., Bagul A., et al. CheXNet: radiologist-level pneumonia detection on chest X-rays with deep learning. arXiv 2017; 1711.05225.
  14. Wang X, Peng Y, Lu L, Lu Z, Bagheri M., et al. ChestX-ray8: hospital-scale chest X-ray database and benchmarks on weakly supervised classification of common thorax diseases. In: IEEE 2017 Conference on Computer Vision and Pattern Recognition; 21-26 July 2017; Honolulu, HI, USA. New York, NY, USA: IEEE. pp. 2097-2106.
  15. Nettur S, Karpurapu S, Nettur U, Gajja L, Myneni S., Dusi A., Posham L. Lightweight weighted average ensemble model for detection pneumonia in chest X-ray images. Comput Artif Intell 2025; 3(2): 3104.
  16. Varshney S, Vajpayee A, Kaur B, Kukana P. Leveraging the PneuNet deep learning model to effectively identify pneumonia from chest X-rays. In: 2025 International Conference on Computing and Communication; 2025. pp. 1-6.
  17. Zech JR, Badgeley MA, Liu M, Costa AB, Titano JJ., Oermann EK. Variable generalization performance of a deep learning model to detect pneumonia in chest radiographs: a cross-sectional study. PLoS Med 2018; 15(11): e1002683.
  18. Xin KZ, Li D, Yi PH. Limited generalizability of deep learning algorithm for pediatric pneumonia classification on external data. J Am Coll Emerg Physicians Open 2021; 2(4): e12554.
  19. Li Y, Zhang Z, Dai C, Dong Q, Badrigilan S. Accuracy of deep learning for automated detection of pneumonia using chest X-ray images: a systematic review and meta-analysis. Comput Biol Med 2020; 123: 103898.
  20. Hofmeister J, Garin N, Montet X, Scheffler M, Platon A., Poletti PA., Stirnemann J., Debray MP., et al. Validating the accuracy of deep learning for the diagnosis of pneumonia on chest x-ray against a robust multimodal reference diagnosis. Eur Radiol Exp 2024; 8: 2.
  21. Meedeniya D, Kumarasinghe H, Kolonne S, Fernando C, Diez IT., Marques G. Chest X-ray analysis empowered with deep learning: a systematic review. BMC Med Imaging 2022; 22: 57.
  22. Yang J, Shi R, Wei D, Liu Z, Zhao L., Ke B., Pfister H., Ni B. MedMNIST v2: a large-scale lightweight benchmark for 2D and 3D biomedical image classification. Sci Data 2023; 10(1): 41.
  23. Shorten C, Khoshgoftaar TM. A survey on image data augmentation for deep learning. J Big Data 2019; 6(1): 60.
  24. Alzubaidi L, Bai J, Al-Sabaawi A, Santamaría J, Albahri AS., Al‑dabbagh BSN., Fadhel MA., Manoufali M., et al. A survey on deep learning tools dealing with data scarcity: definitions, challenges, solutions, tips, and applications. J Big Data 2023; 10(1): 46.
  25. Saporta A, Gui X, Agrawal A, Pareek A, Truong SQH., Nguyen CDT., Ngo VD., Seekins J., et al. Benchmarking saliency methods for chest X-ray interpretation. Nat Mach Intell 2022; 4(10): 867-878.
  26. Kim HE, Cosa-Linan A, Santhanam N, Jannesari M, Maros ME, Ganslandt T. Transfer learning for medical image classification: a literature review. BMC Med Imaging 2022; 22(1): 69.
  27. Zhu Y, Yin Y, Shen Z, Zhao Z, Song H., Wang S., Shen D., Wang Q. UniCAD: Efficient and extendable architecture for multi-task computer-aided diagnosis system. arXiv 2025.
  28. Atık İ. Artık evrişimli sinir ağı kullanılarak göğüs röntgeni görüntülerinde pnömoni tespiti. Gazi Üniversitesi Mühendislik-Mimarlık Fakültesi Dergisi 2024; 39(3): 1719-1732.
  29. Kadiroğlu, Zehra, Erkan Deniz, and Abdurrahman Şenyiğit. Göğüs röntgen görüntülerinde pnömoni tespiti için derin öğrenme modellerinin karşılaştırılması. Gazi Üniversitesi Mühendislik Mimarlık Fakültesi Dergisi. 2023; 39.2: 729-740.
  30. Kermany DS, Goldbaum M, Cai W, Valentim CCS, Liang H, Baxter, SL, McKeown A., Yang G., et al. Identifying medical diagnoses and treatable diseases by image-based deep learning. Cell 2018; 172(5): 1122-1131.e9.
  31. Mazurowski MA, Habas PA, Zurada JM, Lo JY, Baker JA, Tourassi GD. Training neural network classifiers for medical decision making: the effects of imbalanced datasets on classification performance. Neural Netw 2008; 21: 427-436.
  32. Bressem KK, Adams LC, Erxleben C, Hamm B, Niehues SM., Vahldiek JL. Comparing different deep learning architectures for classification of chest radiographs. Sci Rep 2020; 10: 13590.
  33. Ioffe S, Szegedy C. Batch normalization: accelerating deep network training by reducing internal covariate shift. arXiv 2015; 1502.03167.
  34. Kingma DP, Ba J. Adam: a method for stochastic optimization. arXiv 2014; 1412.6980.
  35. Pinckaers H, Litjens G. Training convolutional neural networks with megapixel images. arXiv 2018; 1804.05712.
  36. Troeger, C. E., Blacker, B. F., Khalil, I. A., Zimsen, S. R., Albertson, S. B., Abate, D.,. & Rai, R. K. Mortality, morbidity, and hospitalisations due to influenza lower respiratory tract infections, 2017: an analysis for the Global Burden of Disease Study 2017. Lancet Respir Med 2019; 7(1): 69-89.