SavedAdd your journal

Journals / Black Sea Journal of Engineering and Science / 2026 / Cilt: 9 Sayı: 5

Robustness Evaluation of Hyperspectral Image Classification Models Using Degradation-Aware Metrics

Cem Atılgan(KIRKLARELİ ÜNİVERSİTESİ)0000-0001-7226-0811
Pages
2373–2392
Publication Date
2026-09-15
🏛️Bibliographic metadata and full text access for this article are provided via DergiPark.
DergiPark Record ↗
✨

AI Summary & Key Takeaways

AI Insight
⚡ TL;DR(Too Long; Didn't Read)

The overall accuracy (OA), average accuracy (AA), kappa coefficient, and macro-F1 score were used as standard performance metrics.

📌 3 Key Takeaways

🎯
Objective & Hypothesis:Hyperspectral image classification studies are generally evaluated on clean benchmark datasets and often report high-accuracy values.
📊
Key Findings & Data:However, because real sensor data can be affected by different degradations, such as Gaussian, impulse, stripe, dead-line, and mixed noise, clean-data performance alone may be insufficient to explain the reliability of models under practical conditions.
💡
Conclusion & Impact:The overall accuracy (OA), average accuracy (AA), kappa coefficient, and macro-F1 score were used as standard performance metrics.
🔬Design:Modeling & Simulation
📍Scope:Comprehensive Sample
⚙️Variables:Hyperspectral image classification · Noise robustness · Sensor degradation

Abstract

Hyperspectral image classification studies are generally evaluated on clean benchmark datasets and often report high-accuracy values. However, because real sensor data can be affected by different degradations, such as Gaussian, impulse, stripe, dead-line, and mixed noise, clean-data performance alone may be insufficient to explain the reliability of models under practical conditions. In this study, SVM, k-NN, 1D-CNN, 2D-CNN, 3D-CNN, Hybrid CNN, Dense CNN, and a transformer-based model were evaluated on the Indian Pines, Pavia University, and Salinas datasets under controlled noise scenarios. The overall accuracy (OA), average accuracy (AA), kappa coefficient, and macro-F1 score were used as standard performance metrics. In addition, two robustness-oriented metrics, Relative Degradation Rate (RDR) and Robustness Score (RS), were used to measure noise-induced performance degradation more explicitly. The findings show that model rankings obtained on clean data may change under noisy conditions depending on the dataset and noise type. Performance degradation was more evident for some models, especially under mixed-severity and high-severity noise scenarios. Overall, the results indicate that hyperspectral classification models should be evaluated not only based on clean-data accuracy but also in terms of their robustness against different types of sensor noise.

Özet

Hyperspectral image classification studies are generally evaluated on clean benchmark datasets and often report high-accuracy values. However, because real sensor data can be affected by different degradations, such as Gaussian, impulse, stripe, dead-line, and mixed noise, clean-data performance alone may be insufficient to explain the reliability of models under practical conditions. In this study, SVM, k-NN, 1D-CNN, 2D-CNN, 3D-CNN, Hybrid CNN, Dense CNN, and a transformer-based model were evaluated on the Indian Pines, Pavia University, and Salinas datasets under controlled noise scenarios. The overall accuracy (OA), average accuracy (AA), kappa coefficient, and macro-F1 score were used as standard performance metrics. In addition, two robustness-oriented metrics, Relative Degradation Rate (RDR) and Robustness Score (RS), were used to measure noise-induced performance degradation more explicitly. The findings show that model rankings obtained on clean data may change under noisy conditions depending on the dataset and noise type. Performance degradation was more evident for some models, especially under mixed-severity and high-severity noise scenarios. Overall, the results indicate that hyperspectral classification models should be evaluated not only based on clean-data accuracy but also in terms of their robustness against different types of sensor noise.

🔖

Cite This Article

Atılgan, C. (2026). Robustness Evaluation of Hyperspectral Image Classification Models Using Degradation-Aware Metrics. Black Sea Journal of Engineering and Science, Cilt: 9 Sayı: 5, 2373–2392. https://doi.org/10.34248/bsengineering.1991363

📄 Full Text PDF Viewer

📄

PDF görüntüleyici hazırlanıyor...

References (28)

  1. Ahmad, M., Butt, M. H. F., Usama, M., Altuwaijri, H. A., Mazzara, M., Distefano, S., & Khan, A. M. (2025). Multi-head spatial-spectral mamba for hyperspectral image classification. Remote Sensing Letters, 16(4), 339-353. https://doi.org/10.1080/2150704X.2025.2461330
  2. Ahmad, M., Shabbir, S., Roy, S. K., Hong, D., Wu, X., Yao, J., Khan, A. M., Mazzara, M., Distefano, S., & Chanussot, J. (2022). Hyperspectral image classification: Traditional to deep models: A survey for future prospects. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 15, 968-999. https://doi.org/10.1109/JSTARS.2021.3133021
  3. Bera, S., Shrivastava, V. K., & Satapathy, S. C. (2022). Advances in hyperspectral image classification based on convolutional neural networks: A review. Computer Modeling in Engineering & Sciences, 133(2), 219-250. https://doi.org/10.32604/cmes.2022.020601
  4. Bishop, C. M. (1995). Training with noise is equivalent to Tikhonov regularization. Neural Computation, 7(1), 108-116. https://doi.org/10.1162/neco.1995.7.1.108
  5. Bo, C., Lu, H., & Wang, D. (2018). Spectral-spatial K-nearest neighbor approach for hyperspectral image classification. Multimedia Tools and Applications, 77(9), 10419-10436. https://doi.org/10.1007/s11042-017-4403-9
  6. Ham, J., Chen, Y., Crawford, M. M., & Ghosh, J. (2005). Investigation of the random forest framework for classification of hyperspectral data. IEEE Transactions on Geoscience and Remote Sensing, 43(3), 492-501. https://doi.org/10.1109/TGRS.2004.842481
  7. He, L., Li, J., Liu, C., & Li, S. (2018). Recent advances on spectral-spatial hyperspectral image classification: An overview and new guidelines. IEEE Transactions on Geoscience and Remote Sensing, 56(3), 1579-1597. https://doi.org/10.1109/TGRS.2017.2765364
  8. Hendrycks, D., & Dietterich, T. (2019). Benchmarking neural network robustness to common corruptions and perturbations. International Conference on Learning Representations. https://openreview.net/forum?id=HJz6tiCqYm
  9. Hendrycks, D., Mu, N., Cubuk, E. D., Zoph, B., Gilmer, J., & Lakshminarayanan, B. (2020). AugMix: A simple data processing method to improve robustness and uncertainty. International Conference on Learning Representations. https://openreview.net/forum?id=S1gmrxHFvB
  10. Hong, D., Han, Z., Yao, J., Gao, L., Zhang, B., Plaza, A., & Chanussot, J. (2022). SpectralFormer: Rethinking hyperspectral image classification with transformers. IEEE Transactions on Geoscience and Remote Sensing, 60, 1-15. https://doi.org/10.1109/TGRS.2021.3130716
  11. Li, J., Liang, B., & Wang, Y. (2020). A hybrid neural network for hyperspectral image classification. Remote Sensing Letters, 11(1), 96-105. https://doi.org/10.1080/2150704X.2019.1686780
  12. Li, S., Song, W., Fang, L., Chen, Y., Ghamisi, P., & Benediktsson, J. A. (2019). Deep learning for hyperspectral image classification: An overview. IEEE Transactions on Geoscience and Remote Sensing, 57(9), 6690-6709. https://doi.org/10.1109/TGRS.2019.2907932
  13. Lodhi, V., Chakravarty, D., & Mitra, P. (2019). Hyperspectral imaging system: Development aspects and recent trends. Sensing and Imaging, 20, Article 35. https://doi.org/10.1007/s11220-019-0257-8
  14. Ma, J., Zou, Y., Tang, X., Zhang, X., Liu, F., & Jiao, L. (2024). Spatial pooling transformer network and noise-tolerant learning for noisy hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 62, 1-19. https://doi.org/10.1109/TGRS.2024.3368079
  15. Mei, S., Lian, J., Wang, X., Su, Y., Ma, M., & Chau, L.-P. (2024). A comprehensive study on the robustness of deep learning-based image classification and object detection in remote sensing: Surveying and benchmarking. Journal of Remote Sensing, 4, 0219. https://doi.org/10.34133/remotesensing.0219
  16. Melgani, F., & Bruzzone, L. (2004). Classification of hyperspectral remote sensing images with support vector machines. IEEE Transactions on Geoscience and Remote Sensing, 42(8), 1778-1790. https://doi.org/10.1109/TGRS.2004.831865
  17. Parra, L. (2022). Remote sensing and GIS in environmental monitoring. Applied Sciences, 12(16), 8045. https://doi.org/10.3390/app12168045
  18. Rasti, B., Scheunders, P., Ghamisi, P., Licciardi, G., & Chanussot, J. (2018). Noise reduction in hyperspectral imagery: Overview and application. Remote Sensing, 10(3), 482. https://doi.org/10.3390/rs10030482
  19. Roy, S. K., Krishna, G., Dubey, S. R., & Chaudhuri, B. B. (2020). HybridSN: Exploring 3-D-2-D CNN feature hierarchy for hyperspectral image classification. IEEE Geoscience and Remote Sensing Letters, 17(2), 277-281. https://doi.org/10.1109/LGRS.2019.2918719
  20. Rusak, E., Schott, L., Zimmermann, R. S., Bitterwolf, J., Bringmann, O., Bethge, M., & Brendel, W. (2020). A simple way to make neural networks robust against diverse image corruptions. In A. Vedaldi, H. Bischof, T. Brox, & J.-M. Frahm (Eds.), Computer Vision – ECCV 2020 (Vol. 12348). Springer. https://doi.org/10.1007/978-3-030-58580-8_4
  21. Schneider, S., Rusak, E., Eck, L., Bringmann, O., Brendel, W., & Bethge, M. (2020). Improving robustness against common corruptions by covariate shift adaptation. Advances in Neural Information Processing Systems, 33, 11539-11551. https://proceedings.neurips.cc/paper/2020/hash/85690f81aadc1749175c187784afc9ee-Abstract.html
  22. Stuart, M. B., Davies, M., Hobbs, M. J., Pering, T. D., McGonigle, A. J. S., & Willmott, J. R. (2022). High-resolution hyperspectral imaging using low-cost components: Application within environmental monitoring scenarios. Sensors, 22(12), 4652. https://doi.org/10.3390/s22124652
  23. Sun, L., Zhao, G., Zheng, Y., & Wu, Z. (2022). Spectral-spatial feature tokenization transformer for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 60, 1-14. https://doi.org/10.1109/TGRS.2022.3144158
  24. Sun, L., Zhang, H., Zheng, Y., Wu, Z., Ye, Z., & Zhao, H. (2024). MASSFormer: Memory-augmented spectral-spatial transformer for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 62, 1-15. https://doi.org/10.1109/TGRS.2024.3392264
  25. Xia, J., Du, P., He, X., & Chanussot, J. (2014). Hyperspectral remote sensing image classification based on rotation forest. IEEE Geoscience and Remote Sensing Letters, 11(1), 239-243. https://doi.org/10.1109/LGRS.2013.2254108
  26. Xie, H., Zhang, M., Huang, H., Yang, M., Xu, L., Jiao, Q., & Tan, X. (2026). WMM: Noise-robust hyperspectral image classification via joint wavelet-manifold Mamba framework. Journal of King Saud University - Computer and Information Sciences, 38(4), 155. https://doi.org/10.1007/s44443-026-00538-4
  27. Yu, X., Ozdemir, M. B., & Joshie, M. K. (2023). An overview of hyperspectral image classification by data-driven deep learning. Frontiers in Computing and Intelligent Systems, 5(3), 107-110. https://doi.org/10.54097/fcis.v5i3.13999
  28. Zhi, L., Yu, X., Liu, B., & Wei, X. (2019). A dense convolutional neural network for hyperspectral image classification. Remote Sensing Letters, 10(1), 59-66. https://doi.org/10.1080/2150704X.2018.1526424