Transfer Learning Based Myanmar Sign Language Recognition for Myanmar Consonants
Keywords:
Myanmar Sign Language, Myanmar Fingerspelling, Transfer Learning, Myanmar consonantsAbstract
In this paper, a study on the different Transfer Learning models is made for the purpose of recognizing Myanmar Fingerspelling (Myanmar Sign Language) alphabets. This experiment shows that Transfer Learning can play a significant role for sign language recognition system and is capable of recognizing the static hand gesture images that represent the Myanmar consonants from က (ka) to အ (a). The main objective of this paper is to investigate the performance of various Transfer Learning models for Myanmar Fingerspelling recognition. We proposed 12 Transfer Learning models using TensorFlow library and the accuracy for each model is compared. Among these 12 models, VGG16, ResNet50 and MobileNet with epoch 50 yielded the highest accuracy score with 94%. Although there are some limitations in the datasets, each model provides the encouraging results and thus, it can believe that the fully generalizable recognition system based on Transfer Learning can be produced for all Myanmar Sign Language Fingerspelling characters by doing further research with more data.
References
Win, S. T. (2021). Status of Disability in Myanmar (Doctoral dissertation, MERAL Portal).
Steinberg, I., London, T. M., & Di Castro, D. (2010). Hand gesture recognition in images and video. Irwin and Joan Jacobs Center for Communication and Information Technologies, CCIT Report, 763, 1-20.
Y. Y. Swe, Myanmar Sign Language Basic Conversation Book, 1st Edition ed., Department of Social Welfare, Ministry of Social Welfare, Relief and Resettlement, Department of Social Welfare, Japan International Cooperation Agency, August 2009.
Moe, S. Z., Thu, Y. K., Thant, H. A., Min, N. W., & Supnithi, T. (2020). Unsupervised neural machine translation between myanmar sign language and myanmar language (Doctoral dissertation, MERAL Portal).
Thu, Y. K., Maung, S. A. W., & Urano, Y. (2009). Direct Keyboard Mapping (DKM) layout for Myanmar fingerspelling text input: study with developed fingerspelling font" mmFingerspelling. ttf". GITS, GITI research bulletin= GITS, GITI 紀要, 2009, 127-135.
Moe, S. Z., Thu, Y. K., Nwe, H. M., Hlaing, H. W. W., Aung, N. H., Wai, K. H., ... & Min, N. W. Development of Natural Language Processing based Communication and Educational Assisted Systems for the People with Hearing Disability in Myanmar (Doctoral dissertation, MERAL Portal).
W. Wah, Myanmar Sign Language Recognition System Using Artificial Neural Network, December, 2014.
ThiriMin,ThandaAung,”Video Based Myanmar Sign Language Recognition System,” in 12th National Conference on Science and Engineering, Mandalay, Myanmar, 2019.
Ni Htwe Aung, Ye Kyaw Thu, Su Su Maung, ”Feature Based Myanmar Fingerspelling Image Classification Using SIFT, SURF and BRIEF,” in In Proceedings of the 17th International Confer- ence on Computer Applications (ICCA), Yangon, February, 2019.
Ni Htwe Aung, Su Su Maung, Ye Kyaw Thu, ”Sign Language Recognition for Myanmar Number Using Three Different SVM Classifiers,” in 12th National Conference on Science and Engineering, Yangon, June, 2019.
S.ThrunandL.Pratt,Learningtolearn,L.P.SebastianThrun, Ed., USA: Kluwer Academic Publishers„ 1998.
R. Caruana, ”Multitask learning,” in Machine Learning, vol. 28(1), Netherlands, KluwerAcademicPublishers, 1997, pp. 41-75. [13] Ying Lu, Transfer Learning for Image Classification, Université de Lyon, 2017, p. 71.
Kumar, A., & Daume III, H. (2012). Learning task grouping and overlap in multi-task learning. arXiv preprint arXiv:1206.6417.
Yu Kong, Ming Shao, Kang Li and Yun Fu, ”Probabilistic LowRank Multitask Learning,” in IEEE Transactions on Neural
Networks Learning Systems, 2017.
”ImageNet Large Scale Visual Recognition Challenge 2014 (ILSVRC2014),” [Online]. Available: http://www.imagenet.org/challenges/LSVRC/2014/results.
Karen Simonyan and Andrew Zisserman, Visual Geometry Group, ”Very Deep Convolutional Networks For Large-Scale Image Recognition,” in International Conference on Learning Representations, 2015.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun, ”Deep Residual Learning for Image Recognition,” 2015.
Kaiming He, Xiangyu Zhang,Shaoqing Ren, Jian Sun, ”Deep Residual Learning for Image Recognition,” in In Proceedings of the IEEE conference on computer vision and pattern recognition, 2016.
Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed,Dragomir Anguelov, Dumitru Erhan, Vincent Van- houcke, Andrew Rabinovich, ”Going deeper with convolutions,” in eprint arXiv 1409.4842, 2014
