SavedAdd your journal

Journals / Gazi Üniversitesi Fen Bilimleri Dergisi Part C: Tasarım ve Teknoloji / 2026 / Cilt: 14 Sayı: 3

Multimodal Fusion of Electrocardiogram Signals and Demographic Characteristics with Cross-Attention Networks in the Classification of Cardiovascular Diseases

Kardiyovasküler Hastalıkların Sınıflandırılmasında Elektrokardiyogram Sinyallerinin ve Demografik Özelliklerin Çapraz Dikkat Ağları ile Çok Modlu Birleştirilmesi

Talip Çay(YOZGAT BOZOK ÜNİVERSİTESİ)0000-0002-5490-1651
Emre Ölmez(İZMİR BAKIRÇAY ÜNİVERSİTESİ)0000-0003-1686-0251
Pages
1379–1398
Publication Date
2026-09-30
🏛️Bibliographic metadata and full text access for this article are provided via DergiPark.
DergiPark Record ↗
✨

AI Summary & Key Takeaways

AI Insight
⚡ TL;DR(Too Long; Didn't Read)

Heatmaps generated by overlaying attention weights onto the ECG signal provide temporal interpretability of the decision mechanism.

📌 3 Key Takeaways

🎯
Objective & Hypothesis:Cardiovascular diseases are the leading cause of mortality worldwide.
📊
Key Findings & Data:In the second component, 12-channel ECG recordings were decomposed into multi-scale representations through parallel convolution (1D-CNN), and temporal dependencies were modeled using xLSTM (mLSTM) layers.
💡
Conclusion & Impact:Heatmaps generated by overlaying attention weights onto the ECG signal provide temporal interpretability of the decision mechanism.
🔬Design:Modeling & Simulation
📍Scope:Comprehensive Sample
⚙️Variables:Derin Öğrenme · EKG · Kardiyovasküler Hastalık

Abstract

Cardiovascular diseases are the leading cause of mortality worldwide. In diagnosis, low-cost and non-invasive electrocardiography (ECG) is the primary tool. Since manual examination imposes a high workload on experts, numerous deep learning approaches have been proposed in the literature. In this study, the simultaneous classification of five disease classes (NORM, MI, STTC, CD, HYP) defined by SCP codes in the PTB-XL database was targeted. The proposed architecture consists of three components. In the first component, a second-order polynomial cross-feature transformation was applied to the variables of sex, age, height, and weight. The data were projected into a dmodel-dimensional space using a neural network with a GELU activation function. In the second component, 12-channel ECG recordings were decomposed into multi-scale representations through parallel convolution (1D-CNN), and temporal dependencies were modeled using xLSTM (mLSTM) layers. In the third component, predictions were generated using class-specific context vectors obtained by fusing clinical data as the “query” and ECG data as the “key/value” through cross-attention. In training, which prevented information leakage, the BCEWithLogitsLoss loss function compatible with multi label classification was used. The learning rate and AdamW weight decay were optimized through a 12-cell grid search, ablation studies were conducted, and the model was compared with ResNet and InceptionTime. The 0.9106 macro AUROC and 0.7151 macro F1 scores obtained within a 95% confidence interval across five independent seeds demonstrate the performance of the method. Heatmaps generated by overlaying attention weights onto the ECG signal provide temporal interpretability of the decision mechanism.

Özet

Kardiyovasküler hastalıklar dünya çapında mortalitenin başlıca nedenidir. Tanıda düşük maliyetli ve invaziv olmayan elektrokardiyografi (EKG) birincil araçtır. Manuel inceleme uzmanlara yüksek iş yükü getirdiğinden literatürde çok sayıda derin öğrenme yaklaşımı bulunmaktadır. Bu çalışmada, PTB-XL veri tabanındaki SCP kodlarıyla tanımlanan beş hastalık sınıfının (NORM, MI, STTC, CD, HYP) eş zamanlı sınıflandırılması hedeflenmiştir. Önerilen mimari üç bileşenlidir. İlk bileşende cinsiyet, yaş, boy ve kilo değişkenlerine ikinci dereceden polinom çapraz öznitelik dönüşümü uygulanmıştır. Veriler GELU fonksiyonlu sinir ağıyla dmodel boyutlu uzaya aktarılmıştır. İkinci bileşende 12 kanallı EKG kayıtları paralel evrişimle (1D-CNN) çok ölçekli temsillere ayrılmış, zamansal bağımlılıklar xLSTM (mLSTM) katmanlarıyla modellenmiştir. Üçüncü bileşende klinik veriler 'sorgu', EKG verileri 'anahtar/değer' olarak çapraz dikkatle harmanlanmış sınıfa özgü bağlam vektörleriyle tahmin üretilmiştir. Bilgi sızıntısını önleyen eğitimde çoklu etiketle uyumlu BCEWithLogitsLoss kaybı kullanılmıştır. Öğrenme oranı ve AdamW ağırlık bozumu 12 hücreli ızgara taramasıyla optimize edilmiş, ablasyon çalışmaları yürütülmüş, model ResNet ve InceptionTime ile kıyaslanmıştır. Beş bağımsız tohumda %95 güven aralığında elde edilen 0.9106 makro AUROC ve 0.7151 makro F1 skorları yöntemin başarımını ortaya koymaktadır. Dikkat ağırlıkları EKG sinyaline bindirilerek üretilen ısı haritaları karar mekanizmasına zamansal yorumlanabilirlik kazandırmaktadır.

🔖

Cite This Article

Çay, T., Ölmez, E. (2026). Multimodal Fusion of Electrocardiogram Signals and Demographic Characteristics with Cross-Attention Networks in the Classification of Cardiovascular Diseases. Gazi Üniversitesi Fen Bilimleri Dergisi Part C: Tasarım ve Teknoloji, Cilt: 14 Sayı: 3, 1379–1398. https://doi.org/10.29109/gujsc.1958154

📄 Full Text PDF Viewer

📄

PDF görüntüleyici hazırlanıyor...

References (25)

  1. World Health Organization, "Cardiovascular diseases (CVDs)," 2015. [Online]. Available: http://www.who.int/cardiovascular_diseases/about_cvd/en/. [Accessed: May 11, 2026].
  2. P. Kligfield et al., "Recommendations for the standardization and interpretation of the electrocardiogram: Part I: The electrocardiogram and its technology: A scientific statement from the American Heart Association Electrocardiography and Arrhythmias Committee, Council on Clinical Cardiology," Circulation, vol. 115, no. 10, pp. 1306–1324, Mar. 2007.
  3. A. Lyon, A. Mincholé, J. P. Martínez, P. Laguna, and B. Rodriguez, "Computational techniques for ECG analysis and interpretation in light of their contribution to medical advances," J. R. Soc. Interface, vol. 15, no. 138, 2018.
  4. P. Wagner, N. Strodthoff, R.-D. Bousseljot, et al., "PTB-XL, a large publicly available electrocardiography dataset," Sci. Data, vol. 7, no. 154, pp. 1-15, May 2020, doi: 10.1038/s41597-020-0495-6.
  5. J. Pan and W. J. Tompkins, "A real-time QRS detection algorithm," IEEE Transactions on Biomedical Engineering, vol. 32, no. 3, pp. 230-236, 1985.
  6. R. J. Martis et al., "Application of higher order cumulant features for cardiac health diagnosis using ECG signals," International Journal of Neural Systems, vol. 23, no. 04, p. 1350014, 2013.
  7. E. J. da S. Luz et al., "ECG-based heartbeat classification for arrhythmia detection: A survey," Computer Methods and Programs in Biomedicine, vol. 127, pp. 144-164, 2016.
  8. A. Y. Hannun et al., "Cardiologist-level arrhythmia detection and classification in ambulatory electrocardiograms using a deep convolutional neural network," Nature Medicine, vol. 25, no. 1, pp. 65-69, 2019.
  9. A. H. Ribeiro et al., "Automatic diagnosis of the 12-lead ECG using a deep neural network," Nature Communications, vol. 11, no. 1, p. 1760, 2020.
  10. N. Strodthoff et al., "Deep learning for ECG analysis: Benchmarks and insights from PTB-XL," IEEE Journal of Biomedical and Health Informatics, vol. 25, no. 5, pp. 1519-1528, 2020.
  11. S. Smigiel, K. Pałczyński, and D. Ledziński, "ECG signal classification using deep learning techniques based on the PTB-XL dataset," Entropy, vol. 23, no. 9, p. 1121, 2021.
  12. T. Mehari and N. Strodthoff, "Self-supervised representation learning from 12-lead ECG data," Computers in Biology and Medicine, vol. 141, p. 105114, 2022.
  13. Ö. Yıldırım, "A novel wavelet sequence based on deep bidirectional LSTM network model for ECG signal classification," Computers in Biology and Medicine, vol. 96, pp. 189-202, 2018.
  14. Z. Wang et al., "A deep learning framework for the detection of abnormal heart rhythms," IEEE Access, vol. 7, pp. 121543-121553, 2019.
  15. M. Beck et al., "xLSTM: Extended Long Short-Term Memory," arXiv preprint arXiv:2405.04517, 2024.
  16. A. Vaswani et al., "Attention is all you need," Advances in Neural Information Processing Systems, vol. 30, 2017.
  17. J. Song et al., "Transformer-Based Spatial-Temporal Feature Learning for ECG Arrhythmia Classification," IEEE Journal of Biomedical and Health Informatics, vol. 26, no. 10, pp. 5001-5012, Oct. 2022.
  18. C. Che et al., "Constrained transformer network for ECG signal processing and arrhythmia classification," BMC Medical Informatics and Decision Making, vol. 21, 2021.
  19. S. C. Huang et al., "Fusion of medical imaging and electronic health records using deep learning: a systematic review and implementation guidelines," NPJ Digital Medicine, vol. 3, no. 136, 2020.
  20. C. Qu, X. Zhang, Y. Lu, Y. Wang, and C. Su, "MAF-Net: Multimodal cross-attention-based fusion network for cardiovascular disease classification," PLoS One, vol. 21, no. 4, Art. no. e0345238, Apr. 2026, doi: 10.1371/journal.pone.0345238.
  21. S. Kiranyaz, O. Avci, O. Abdeljaber, T. Ince, M. Gabbouj, and D. J. Inman, "1D convolutional neural networks and applications: A survey," Mechanical Systems and Signal Processing, vol. 151, p. 107398, 2021, doi: 10.1016/j.ymssp.2020.107398.
  22. J. Qu, Z. Lin, X. Liang, and J. Song, "Decomposing xLSTM: A comprehensive study on financial time series forecasting," Mathematics, vol. 14, no. 8, Art. no. 1282, Apr. 2026, doi: 10.3390/math14081282.
  23. T. Baltrušaitis, C. Ahuja, and L.-P. Morency, "Multimodal machine learning: A survey and taxonomy," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 41, no. 2, pp. 423-443, Feb. 2019, doi: 10.1109/TPAMI.2018.2798607.
  24. P. Xu, X. Zhu, and D. A. Clifton, "Multimodal learning with transformers: A survey," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 45, no. 10, pp. 12113-12132, Oct. 2023, doi: 10.1109/TPAMI.2023.3275156.
  25. K. Han et al., "A Survey on Vision Transformer," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 45, no. 1, pp. 87-110, Jan. 2023, doi: 10.1109/TPAMI.2022.3152247.