SavedAdd your journal

Journals / Afyon Kocatepe Üniversitesi Fen ve Mühendislik Bilimleri Dergisi / 2026 / Cilt: 24 Sayı: 5

HGNN: A Hybrid Graph Neural Network Based on Transfer Learning for Linguistic Steganalysis

HGNN: Dilsel Steganaliz için Transfer Öğrenimine Dayalı Hibrit Grafik Sinir Ağı

Merve Varol Arısoy(Burdur Mehmet Akif Ersoy Üniversitesi)0000-0003-2085-1964
Pages
1138–1149
Publication Date
2024-10-01
🏛️Bibliographic metadata and full text access for this article are provided via DergiPark.
DergiPark Record ↗
✨

AI Summary & Key Takeaways

AI Insight
⚡ TL;DR(Too Long; Didn't Read)

Experiments demonstrates that the proposed model surpass other methods in terms of the accuracy (a success of 0.9879 at 3.22 bpw on text encoded through SAAC Encoding) and the effectiveness of inference (1.09 second).

📌 3 Key Takeaways

🎯
Objective & Hypothesis:Steganography, especially in the form of text generation based on secret messages, has become a current research topic.
📊
Key Findings & Data:Owing to the high rate of imperceptibility and resistance to steganalysis of this type steganography, it is essential that steganalysis methods, generate better performance.
💡
Conclusion & Impact:Experiments demonstrates that the proposed model surpass other methods in terms of the accuracy (a success of 0.9879 at 3.22 bpw on text encoded through SAAC Encoding) and the effectiveness of inference (1.09 second).
🔬Design:Modeling & Simulation
📍Scope:Comprehensive Sample
⚙️Variables:BERT · GAT (Graph Attention Network) · GCN (Graph Convolutional Networks)

Abstract

Steganography, especially in the form of text generation based on secret messages, has become a current research topic. It is more difficult to identify the hidden message when it embedded directly into the text without using a cover text, and it also has a higher embedding capacity. Owing to the high rate of imperceptibility and resistance to steganalysis of this type steganography, it is essential that steganalysis methods, generate better performance. Although the complexity of deep learning models increases the accuracy rate, it also increases the inference time. In this study, a linguistic steganalysis was performed with a lower inference time and a higher accuracy rate. In the developed model, first, differences between non-stega and steganographic texts were modelled by a finetuned Bert using the custom dataset. The disparity information obtained by fine-tuned model was distilled into 3 separate networks, BertGCN, BertGAT and BertGIN, for faster and more accurate inference. Then, these 3 distilled networks were combined through Transfer Learning to form a new model. Experiments demonstrates that the proposed model surpass other methods in terms of the accuracy (a success of 0.9879 at 3.22 bpw on text encoded through SAAC Encoding) and the effectiveness of inference (1.09 second).

Özet

Özellikle gizli mesajlara dayalı metin üretimi şeklindeki steganografi güncel bir araştırma konusu haline gelmiştir. Gizli mesajın kapak metni kullanılmadan doğrudan metnin içine gömülmesi durumunda tespit edilmesi daha zor olduğu gibi gömme kapasitesi de daha yüksektir. Bu tip steganografinin algılanamazlık oranının yüksek olması ve steganalize karşı direnci nedeniyle, steganaliz yöntemlerinin daha iyi performans üretmesi önemlidir. Derin öğrenme modellerinin karmaşıklığı doğruluk oranını arttırsa da çıkarım süresini de arttırmaktadır. Bu çalışmada, daha düşük çıkarım süresi ve daha yüksek doğruluk oranıyla dilsel steganaliz gerçekleştirilmiştir. Geliştirilen modelde öncelikle stega olmayan ve steganografik metinler arasındaki farklar, özel veri seti kullanılarak hassas ayarlı Bert tarafından modellendi. İnce ayarlı modelle elde edilen eşitsizlik bilgisi, daha hızlı ve daha doğru çıkarım için BertGCN, BertGAT ve BertGIN olmak üzere 3 ayrı ağa ayrıştırıldı. Daha sonra bu 3 damıtılmış ağ, Transfer Öğrenme yoluyla birleştirildi ve yeni bir model oluşturuldu. Deneyler, önerilen modelin doğruluk (SAAC Kodlama yoluyla kodlanan metinde 3,22 bpw'de 0,9879 başarı) ve çıkarımın etkinliği (1,09 saniye) açısından diğer yöntemleri geride bıraktığını göstermektedir.

🔖

Cite This Article

Arısoy, M. V. (2024). HGNN: A Hybrid Graph Neural Network Based on Transfer Learning for Linguistic Steganalysis. Afyon Kocatepe Üniversitesi Fen ve Mühendislik Bilimleri Dergisi, Cilt: 24 Sayı: 5, 1138–1149. https://doi.org/10.35414/akufemubid.1427644

📄 Full Text PDF Viewer

📄

PDF görüntüleyici hazırlanıyor...

References (37)

  1. Chen, Z., Huang, L., Miao, H., Yang, W., Meng, P. 2011. Steganalysis against substitution-based linguistic steganography based on context clusters. Computers & Electrical Engineering, 37(6), 1071-1081. https://doi.org/10.1016/j.compeleceng.2011.09.014
  2. Fang, T., Jaggi, M., Argyraki, K. 2017. Generating steganographic text with LSTMs. arXiv preprint arXiv:1705.10742. https://doi.org/10.48550/arXiv.1705.10742
  3. Fu, Z., Yu, Q., Wang, F., Ding, C. 2022. HGA: Hierarchical feature extraction with graph and attention mechanism for linguistic steganalysis. IEEE Signal Processing Letters, 29, 1734-1738. https://doi.org/10.1109/LSP.2022.3192534
  4. Jing, W., Song, X., Di, D., Song, H. 2021. GeoGAT: Graph model based on attention mechanism for geographic text classification. Transactions on Asian and Low-Resource Language Information Processing, 20(5), 1-18. https://doi.org/10.1145/3450626
  5. Kang, H., Wu, H., Zhang, X. 2020. Generative text steganography based on LSTM network and attention mechanism with keywords. Electronic Imaging, 2020(4), 291-1. https://doi.org/10.2352/ISSN.2470-1173.2020.4.MWSF-291
  6. Kingma, D.P., Ba, J. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980. https://doi.org/10.48550/arXiv.1412.6980
  7. Kipf, T.N., Welling, M. 2016. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907. https://doi.org/10.48550/arXiv.1609.02907
  8. Li, S., & Wang, J., Liu, P. 2022. Detection of generative linguistic steganography based on explicit and latent text word relation mining using deep learning. IEEE Transactions on Dependable and Secure Computing, 20(2),1476-148. https://doi.org/10.1109/TDSC.2021.3062703
  9. Lin, Y., Meng, Y., Sun, X., Han, Q., Kuang, K., Li, J., Wu, F. 2021. BertGCN: Transductive text classification by combining GCN and BERT. arXiv preprint arXiv:2105.05727. https://doi.org/10.48550/arXiv.2105.05727
  10. Liu, P., Tian, B., Liu, X., Gu, S., Yan, L., Bullock, L., Zhang, W. 2022. Construction of power fault knowledge graph based on deep learning. Applied Sciences, 12(14), 6993. https://doi.org/10.3390/app12146993
  11. Meng, P., Hang, L., Chen, Z., Hu, Y., Yang, W. (2010). STBS: A statistical algorithm for steganalysis of translation-based steganography. Information Hiding: 12th International Conference. IH 2010. Calgary, AB, Canada, 208-220.
  12. Meng, P., Hang, L., Yang, W., Chen, Z., Zheng, H. (2009). Linguistic steganography detection algorithm using statistical language model. Proceedings of the 2009 International Conference on Information Technology and Computer Science. Kiev, Ukraine, 25-26.
  13. Niu, Y., Wen, J., Zhong, P., Xue, Y. 2019. A hybrid R-BILSTM-C neural network based text steganalysis. IEEE Signal Processing Letters, 26(12), 1907-1911. https://doi.org/10.1109/LSP.2019.2955374
  14. Peng, W., Zhang, J., Xue, Y., Yang, Z. 2021. Real-time text steganalysis based on multi-stage transfer learning. IEEE Signal Processing Letters, 28, 1510-1514. https://doi.org/10.1109/LSP.2021.3105493
  15. Rassil, A., Chougrad, H., Zouaki, H. (2020). The importance of local labels distribution and dominance for node classification in graph neural networks. Proceedings of the 2020 19th IEEE International Conference on Machine Learning and Applications (ICMLA). Miami, FL, USA, pp. 1505-1511.
  16. Shen, J., Heng, J., & Han, J. 2020. Near-imperceptible neural linguistic steganography via self-adjusting arithmetic coding. arXiv preprint arXiv:2010.00677. https://doi.org/10.48550/arXiv.2010.00677
  17. Velickovic, P., Cucurull, G., Casanova, A., Romero, A., Lio, P., Bengio, Y. 2017. Graph attention networks. stat, 1050(20), 10-48550. https://doi.org/10.48550/arXiv.1710.10903
  18. Wang, H., Li, F. 2022. A text classification method based on LSTM and graph attention network. Connection Science, 34(1), 2466-2480. https://doi.org/10.1080/09540091.2022.2044605
  19. Wen, J., Zhou, X., Zhong, P., Xue, Y. 2019. Convolutional neural network based text steganalysis. IEEE Signal Processing Letters, 26(3), 460-464. https://doi.org/10.1109/LSP.2019.2895260
  20. Wu, H., Yi, B., Ding, F., Feng, G., Zhang, X. 2021. Linguistic steganalysis with graph neural networks. IEEE Signal Processing Letters, 28, 558-562. https://doi.org/10.1109/LSP.2021.3058369
  21. Xiang, L., Liu, Y., You, H., Ou, C. 2022. Aggregating local and global text features for linguistic steganalysis. IEEE Signal Processing Letters, 29, 1502-1506. https://doi.org/10.1109/LSP.2022.3190781
  22. Xiang, L., Sun, X., Luo, G., Xia, B. 2014. Linguistic steganalysis using the features derived from synonym frequency. Multimedia Tools and Applications, 71, 1893-1911. https://doi.org/10.1007/s11042-012-1303-4
  23. Xiang, L., Yu, J., Yang, C., Zeng, D., Shen, X. 2018. A word-embedding-based steganalysis method for linguistic steganography via synonym substitution. IEEE Access, 6,64131-64141. https://doi.org/10.1109/ACCESS.2018.2876935
  24. Xu, K., Hu, W., Leskovec, J., Jegelka, S. 2018. How powerful are graph neural networks?. arXiv preprint arXiv:1810.00826. https://doi.org/10.48550/arXiv.1810.00826
  25. Yang, H., Bao, Y., Yang, Z., Liu, S., Huang, Y., Jiao, S. (2020). Linguistic steganalysis via densely connected LSTM with feature pyramid. Proceedings of the 2020 ACM Workshop on Information Hiding and Multimedia Security. 5-10.
  26. Yang, H., Cao, X. 2010. Linguistic steganalysis based on meta features and immune mechanism. Chinese Journal of Electronics, 19, 661-666. https://doi.org/10.1049/cje.2010.661666
  27. Yang, J., Yang, Z., Zhang, S., Tu, H., Huang, Y. 2021. SeSy: linguistic steganalysis framework integrating semantic and syntactic features. IEEE Signal Processing Letters, 29, 31-35. https://doi.org/10.1109/LSP.2021.3131807
  28. Yang, Z., Guo, X., Chen, Z., Huang, Y., Zhang, Y. 2019 (a). RNN-Stega: linguistic steganography based on recurrent neural networks. IEEE Transactions on Information Forensics and Security, 14(5), 1280-1295. https://doi.org/10.1109/TIFS.2018.2871746 (a)
  29. Yang, Z., Huang, Y., Zhang, Y.J. 2019(b). A fast and efficient text steganalysis method. IEEE Signal Processing Letters,26(4),627-631. https://doi.org/10.1109/LSP.2019.2903902 (b)
  30. Yang, Z., Wang, K., Li, J., Huang, Y., Zhang, Y.J. 2019(c). TS-RNN: text steganalysis based on recurrent neural networks. IEEE Signal Processing Letters, 26(12), 1743-1747. https://doi.org/10.1109/LSP.2019.2950464 (c)
  31. Yang, Z., Huang, Y., Zhang, Y.J. 2020. TS-CSW: text steganalysis and hidden capacity estimation based on convolutional sliding windows. Multimedia Tools and Applications, 79, 18293-18316. https://doi.org/10.1007/s11042-019-08345-7
  32. Yang, Z.L., Zhang, S.Y., Hu, Y.T., Hu, Z.W., Huang, Y.F. 2021. VAEStega: linguistic steganography based on variational auto-encoder. IEEE Transactions on Information Forensics and Security, 16, 880-895. https://doi.org/10.1109/TIFS.2020.3037121
  33. Yao, L., Mao, C., Luo, Y. (2019). Graph convolutional networks for text classification. Proceedings of the AAAI Conference on Artificial Intelligence. 7370-7377.
  34. Zhang, L., Ding, J., Xu, Y., Liu, Y., Zhou, S. 2021. Weakly-supervised text classification based on keyword graph. arXiv preprint arXiv:2110.02591. https://doi.org/10.48550/arXiv.2110.02591
  35. Zhang, Y., Xu, Y., Zhang, Y. 2023. A graph neural network node classification application model with enhanced node association. Applied Sciences, 13(12), 7150. https://doi.org/10.3390/app13127150
  36. Ziegler, Z., Deng, Y., Rush, A. (2019). Neural Linguistic Steganography. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). Hong Kong, China, 1210-1215.
  37. Zou, J., Yang, Z., Zhang, S., Rehman, S.U., & Huang, Y. (2020). High-Performance Linguistic Steganalysis, Capacity Estimation and Steganographic Positioning. In Digital Forensics and Watermarking: 19th International Workshop, IWDW 2020. Melbourne, VIC, Australia, 80-93.