SavedAdd your journal

Journals / Uluslararası Mühendislik Araştırma ve Geliştirme Dergisi / 2022 / Cilt: 14 - Sayı: 2

A Survey on Lip-Reading with Deep Learning

Ali ERBEY
Necaattin BARIŞÇI
Pages
844–860
Publication Date
2022-07-31
DOI
—
🏛️Bibliographic metadata and full text access for this article are provided via DergiPark.
DergiPark Record ↗
✨

AI Summary & Key Takeaways

AI Insight
⚡ TL;DR(Too Long; Didn't Read)

In this context, it is seen that technologies that can be used in line with the need can be developed by conducting more academic studies on lip reading.

📌 3 Key Takeaways

🎯
Objective & Hypothesis:Very successful results have been obtained in areas such as computer vision and voice recognition when applying deep learning methods.
📊
Key Findings & Data:Thanks to computerized vision, the success of voice recognition devices can be increased with the analysis of human lips in order to determine what the speaker is saying.
💡
Conclusion & Impact:In this context, it is seen that technologies that can be used in line with the need can be developed by conducting more academic studies on lip reading.
🔬Design:Modeling & Simulation
📍Scope:2017 Verileri
⚙️Variables:Lipreading · Deep Learning · Convolutional Neural Networks

Abstract

Very successful results have been obtained in areas such as computer vision and voice recognition when applying deep learning methods. Technologies that facilitate the lives of people have been developed as a result of the successes of deep learning within these areas. One of these technologies is voice recognition devices. Research has shown that these devices do not give good results in noisy environments; although, they do give good results in silent environments. With deep learning methods, voice recognition in noisy environments can be achieved using visual signals. Thanks to computerized vision, the success of voice recognition devices can be increased with the analysis of human lips in order to determine what the speaker is saying. In this study, lip-reading studies using deep learning methods published between 2017 and 2020 were examined and data sets were introduced. As a result of the study, it is seen that CNN and LSTM architectures are used more intensively in lip-reading studies, hybrid models are preferred more and the success rates are increasing day by day. In this context, it is seen that technologies that can be used in line with the need can be developed by conducting more academic studies on lip reading.

🔖

Cite This Article

ERBEY, A., BARIŞÇI, N. (2022). A Survey on Lip-Reading with Deep Learning. Uluslararası Mühendislik Araştırma ve Geliştirme Dergisi, Cilt: 14 - Sayı: 2, 844–860. https://ajindex.com/articles/889137

📄 Full Text PDF Viewer

📄

PDF görüntüleyici hazırlanıyor...

References (11)

  1. Adeel, A., Gogate, M., & Hussain, A. (2020). Contextual deep learning-based audio-visual switching for speech enhancement in real-world environments. Information Fusion, 59, 163-170.
  2. Afouras, T., Chung, J. S., & Zisserman, A. (2018). Deep lip reading: a comparison of models and an online application. arXiv preprint arXiv:1806.06053. Afouras, T., Chung, J. S., & Zisserman, A. (2018). LRS3-TED: a large-scale dataset for visual speech recognition. arXiv preprint arXiv:1809.00496. Akmese Ö.F., Erbay H., Kör H., (2019). Derin Ögrenme ile Görüntü Kümeleme. In: 5th International Management Information Systems Conference, Ankara.
  3. Alpaydin, E. (2020). Introduction to machine learning. MIT press. Amanullah, M. A., Habeeb, R. A. A., Nasaruddin, F. H., Gani, A., Ahmed, E., Nainar, A. S. M.,. & Imran, M. (2020). Deep learning and big data technologies for IoT security. Computer Communications, 151, 495-517.
  4. Anina, I., Zhou, Z., Zhao, G., & Pietikäinen, M. (2015, May). Ouluvs2: A multi-view audiovisual database for non-rigid mouth motion analysis. In 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG) (Vol. 1, pp. 1-5).
  5. IEEE. Arı, A., & Hanbay, D. (2019). Tumor detection in MR images of regional convolutional neural networks. Journal of the Faculty of Engineering and Architecture of Gazi University, 34(3), 1395-1408.
  6. Bacciu, D., Micheli, A., & Podda, M. (2020). Edge-based sequential graph generation with recurrent neural networks. Neurocomputing, 416, 177-189.
  7. Bayram, F. (2020). Derin öğrenme tabanlı otomatik plaka tanıma. Politeknik Dergisi, 23(4), 955-960.
  8. Bear, H. L., & Harvey, R. (2017). Phoneme-to-viseme mappings: the good, the bad, and the ugly. Speech Communication, 95, 40-67.
  9. Bi, C., Zhang, D., Yang, L., & Chen, P. (2019, November). An Lipreading Modle with DenseNet and E3D-LSTM. In 2019 6th International Conference on Systems and Informatics (ICSAI) (pp. 511-515).
  10. IEEE. Bollier, D. (2017). Artificial intelligence comes of age. The promise and challenge of integrating AI into cars, healthcare and journalism. The Aspen Institute Communications and Society Program. Washington, DC. Chen, L., Xu, G., Zhang, S., Yan, W., & Wu, Q. (2020). Health indicator construction of machinery based on end-to-end trainable convolution recurrent neural networks. Journal of Manufacturing Systems, 54, 1-11.
  11. Chen, X., Du, J., & Zhang, H. (2020). Lipreading with DenseNet and resBi-LSTM. Signal, Image and Video Processing, 14(5), 981-9