Journals / International Journal of Intelligent Systems and Applications in Engineering / 2022 / Cilt: 10 - Sayı: 1

Detecting Face-Touch Hand Moves Using Smartwatch Inertial Sensors and Convolutional Neural Networks

Pages
122–128
Publication Date
2022-10-18
DOI
—
🏛️Bibliographic metadata and full text access for this article are provided via DergiPark.
DergiPark Record ↗
✨

AI Summary & Key Takeaways

AI Insight
⚡ TL;DR(Too Long; Didn't Read)

Achieved results show state of the art face-touch accuracy with average recall, precision, and F1-Score of 96.75%, 95.1%, 95.85% respectively, with low False Positives Rate (FPR) as 0.04%.

📌 3 Key Takeaways

🎯
Objective & Hypothesis:As per World Health Organization (WHO), avoiding touching the face when people are in public or crowded places is an effective way to prevent respiratory viral infections.
📊
Key Findings & Data:This recommendation has become more crucial with the current health crisis and the worldwide spread of COVID-19 pandemic.
💡
Conclusion & Impact:Achieved results show state of the art face-touch accuracy with average recall, precision, and F1-Score of 96.75%, 95.1%, 95.85% respectively, with low False Positives Rate (FPR) as 0.04%.
🔬Design:Modeling & Simulation
📍Scope:Comprehensive Sample
⚙️Variables:Domain Parameters

Abstract

As per World Health Organization (WHO), avoiding touching the face when people are in public or crowded places is an effective way to prevent respiratory viral infections. This recommendation has become more crucial with the current health crisis and the worldwide spread of COVID-19 pandemic. However, most face touches are done unconsciously, that is why it is difficult for people to monitor their hand moves and try to avoid touching the face all the time. Hand-worn wearable devices like smartwatches are equipped with multiple sensors that can be utilized to track hand moves automatically. This work proposes a smartwatch application that uses small, efficient, and end-to-end Convolutional Neural Networks (CNN) models to classify hand motion and identify Face-Touch moves. To train the models, a large dataset is collected for both left and right hands with over 28k training samples that represents multiple hand motion types, body positions, and hand orientations. The app provides real-time feedback and alerts the user with vibration and sound whenever attempting to touch the face. Achieved results show state of the art face-touch accuracy with average recall, precision, and F1-Score of 96.75%, 95.1%, 95.85% respectively, with low False Positives Rate (FPR) as 0.04%. By using efficient configurations and small models, the app achieves high efficiency and can run for long hours without significant impact on battery which makes it applicable on most off-the-shelf smartwatches.

🔖

Cite This Article

SEHİRLİ, E., ALESMAEİL, A. (2022). Detecting Face-Touch Hand Moves Using Smartwatch Inertial Sensors and Convolutional Neural Networks. International Journal of Intelligent Systems and Applications in Engineering, Cilt: 10 - Sayı: 1, 122–128. https://ajindex.com/articles/541262

📄 Full Text PDF Viewer

📄

PDF görüntüleyici hazırlanıyor...

References (13)

  1. Y. L. A. Kwok, J. Gralton, and M. L. McLaws, “Face touching: A frequent habit that has implications for hand hygiene,” Am. J. Infect. Control, vol. 43, no. 2, pp. 112–114, 2015, doi: 10.1016/j.ajic.2014.10.015
  2. D. Moazen, S. A. Sajjadi, and A. Nahapetian, “AirDraw: Leveraging Smart Watch Motion Sensors for Mobile Human Computer Interactions,” arXiv, pp. 1–5, 2017
  3. M. C. Kwon, G. Park, and S. Choi, “Smartwatch user interface implementation using CNN-based gesture pattern recognition,” Sensors (Switzerland), vol. 18, no. 9, pp. 1–12, 2018, doi: 10.3390/s18092997
  4. M. Kim, J. Cho, S. Lee, and Y. Jung, “Imu sensor-based hand gesture recognition for human-machine interfaces,” Sensors (Switzerland), vol. 19, no. 18, pp. 1–13, 2019, doi: 10.3390/s19183827
  5. J. Hou et al., “SignSpeaker: A real-time, high-precision smartwatchbased sign language translator,” Proc. Annu. Int. Conf. Mob. Comput. Networking, MOBICOM, 2019, doi: 10.1145/3300061.3300117
  6. Q. Zhang, D. Wang, R. Zhao, and Y. Yu, “MyoSign,” pp. 650–660, 2019, doi: 10.1145/3301275.3302296
  7. D. Ekiz et al., “Sign sentence recognition with smart watches,” 2017 25th Signal Process. Commun. Appl. Conf. SIU 2017, 2017, doi: 10.1109/SIU.2017.7960255
  8. S. Taghavi, F. Davari, H. T. Malazi, and A. Ali Abin, “Tennis stroke detection using inertial data of a smartwatch,” 2019 9th Int. Conf. Comput. Knowl. Eng. ICCKE 2019, no. Iccke, pp. 466–474, 2019, doi: 10.1109/ICCKE48569.2019.8964775
  9. A. Anand, M. Sharma, R. Srivastava, L. Kaligounder, and D. Prakash, “Wearable motion sensor based analysis of swing sports,” 2017, doi: 10.1109/ICMLA.2017.0-149
  10. S. S. Tabrizi, S. Pashazadeh, and V. Javani, “Comparative Study of Table Tennis Forehand Strokes Classification Using Deep Learning and SVM,” IEEE Sens. J., vol. 20, no. 22, pp. 13552–13561, 2020, doi: 10.1109/JSEN.2020.3005443
  11. G. Brunner, D. Melnyk, B. Sigfússon, and R. Wattenhofer, “Swimming style recognition and lap counting using a smartwatch and deep learning,” Proc. - Int. Symp. Wearable Comput. ISWC, pp. 23–31, 2019, doi: 10.1145/3341163.3347719
  12. G. Laput and C. Harrison, “Sensing fine-grained hand activity with smartwatches,” Conf. Hum. Factors Comput. Syst. - Proc., 2019, doi: 10.1145/3290605.3300568
  13. K. Kyritsis, C. Diou, and A. Delopoulos, “End-to-end Learning for Measuring in-meal Eating Behavior from a Smartwatch,” Conf. Proc. Annu. Int. Conf. IEEE Eng. Med. Biol. Soc. IEEE Eng. Med. Biol. Soc. Annu. Conf.,