HOSPISIGN: AN INTERACTIVE SIGN LANGUAGE PLATFORM FOR HEARING IMPAIRED

In order to evaluate the developed interface, we performed usability tests and deduced that the system was able to assist its users in real time with high accuracy.

📌 3 Key Takeaways

🎯

Objective & Hypothesis:Sign language is the natural medium of communication for the Deaf community.

📊

Key Findings & Data:The system consists of a computer, a touch display to visualize the interface, and Microsoft Kinect v2 sensor to capture the users’ sign responses.

💡

Conclusion & Impact:In order to evaluate the developed interface, we performed usability tests and deduced that the system was able to assist its users in real time with high accuracy.

🔬Design:Review & Systematic Survey

📍Scope:Comprehensive Sample

⚙️Variables:Domain Parameters

Abstract

Sign language is the natural medium of communication for the Deaf community. In this study, we have developed an interactive communication interface for hospitals, HospiSign, using computer vision based sign language recognition methods. The objective of this paper is to review sign language based Human-Computer Interaction applications and to introduce HospiSign in this context. HospiSign is designed to meet deaf people at the information desk of a hospital and to assist them in their visit. The interface guides the deaf visitors to answer certain questions and express intention of their visit, in sign language, without the need of a translator. The system consists of a computer, a touch display to visualize the interface, and Microsoft Kinect v2 sensor to capture the users’ sign responses. HospiSign recognizes isolated signs in a structured activity diagram using Dynamic Time Warping based classifiers. In order to evaluate the developed interface, we performed usability tests and deduced that the system was able to assist its users in real time with high accuracy.

🔖

Cite This Article

SÜZGÜN, M., ÖZDEMİR, H., CAMGÖZ, N., KINDIROĞLU, A., BAŞARAN, D., TOGAY, C., AKARUN, L. (2016). HOSPISIGN: AN INTERACTIVE SIGN LANGUAGE PLATFORM FOR HEARING IMPAIRED. Journal of Naval Sciences and Engineering, Cilt: 11 - Sayı: 3, 75–92. https://ajindex.com/articles/238053

📄 Full Text PDF Viewer

Open in New Tab ↗

↗New Tab

📄

PDF görüntüleyici hazırlanıyor...

References (7)

H. Cooper, B. Holt, and R. Bowden, (2011).Sign Language Recognition, in Visual Analysis of Humans, Springer, pp. 539–562. L. R. Rabiner, (1989).A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition,Proceedings of the IEEE, vol. 77, no. 2, pp. 257–286.
D. J. Berndt and J. Clifford, Using Dynamic Time Warping to Find Patterns in Time Series,in KDD Workshop, vol. 10, 1994, pp. 359– 370. T. Starner and A. Pentland, (1997).Real-time American Sign Language Recognition from Video Using Hidden Markov Models,in Motion-Based Recognition. Springer, pp. 227–243.
K. Grobel and M. Assan, Isolated Sign Language Recognition using Hidden Markov Models,in IEEE Systems, Man, and Cybernetics International Conference on Computational Cybernetics and Simulation, vol. 1, 1997, pp. 162–167.
C. Vogler and D. Metaxas, Parallel Hidden Markov Models for American Sign Language Recognition,in the Proceedings of the IEEE Seventh International Conference on Computer Vision, vol. 1,1999, pp. 116–122. H.-K. Lee and J. H. Kim,(1999).An HMM-based Threshold Model Approachfor Gesture Recognition, IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), vol. 21, no. 10, pp. 961– 973. X. Chai, G. Li, X. Chen, M. Zhou, G. Wu, and H. Li, Visualcomm: A Tool to Support Communication betweenDeaf and Hearing Persons withthe Kinect, in Proceedings of the 15th International ACM SIGACCESS Conference on Computers and Accessibility. ACM, 2013, p. 76. S. Theodorakis, V. Pitsikalis, and P. Maragos, (2014).Dynamic Static Unsupervised Sequentiality, Statistical Subunits and Lexicon for Sign Language Recognition, Image and Vision Computing, vol. 32, no. 8, pp. 533–549.
V. Pitsikalis, S. Theodorakis, C. Vogler, and P. Maragos,Advances in Phonetics-based Sub-unit Modeling for Transcription Alignment andSign Language Recognition, in IEEE Computer Society Conference onComputer Vision and Pattern Recognition Workshops(CVPRW), 2011. Z. Zhang, (2012). Microsoft Kinect Sensor and Its Effect, MultiMedia, IEEE,vol. 19, no. 2, pp. 4–10. B. S. Parton, (2006). Sign Language Recognition and Translation: A Multidiscipline Approach from the Field of Artificial Intelligence,Journal of Deaf Studies and Deaf Education, vol. 11, no. 1, pp. 94–101.
J. Shotton, T. Sharp, A. Kipman, A. Fitzgibbon, M. Finocchio, A. Blake, M. Cook, and R. Moore, (2013). Real-Time Human Pose Recognition in Parts from Single Depth Images, Communications of the ACM, vol. 56, no. 1, pp. 116–124.
S. Cox, Speech and