Chinese Journal of Computers   Full Text
Title	A Data-Deficiency-Tolerated Method for Viewpoint Independent Sign Language Recognition
Authors	WANG Qi1) CHEN Xi-Lin2) WANG Chun-Li3) GAO Wen2),4)
Address	1)(School of Computer Science and Technology, Harbin Institute of Technology, Harbin 150001) 2)(Key Laboratory of Intelligent Information Processing, Institute of Computing Technology, Chinese Academy of Sciences, Beijing 100190) 3)(School of Information Science & Technology, Dalian Maritime University, Dalian, Liaoning 116026) 4)(School of Electronics Engineering and Computer Science, Peking University, Beijing 100871)
Year	2009
Issue	No.5(953—961)
Abstract &   Background	Abstract This paper proposes a Sample-Consensus method for viewpoint independent sign language recognition under data deficiency(matched features are possibly deficient with regard to some frame pairs). The proposed method is based on the epipolar geometry and inspired by RANSAC. The basic idea is that all corresponded frames between two sequences of the same sign can be roughly considered as captured synchronously by a virtual stereo vision system and thus they will satisfy the same fundamental matrix. In addition, the fundamental matrix can be estimated from point correspondences contained by some part of corresponding frames. Experimental results demonstrate the efficiency of the proposed method. Moreover, this Sample-Consensus method can be easily extended to some similar problems, such as viewpoint independent activity analysis and rigid-motion analysis. Keywords sign language recognition; data deficiency; epipolar constraint; RANSAC Background Sign language recognition aims to translate sign language to text or speech, so as to bridge the communication between the deaf and the hearing people and help the deaf or hard-of-hearing better integrate into the society. According to data collection of sign language, sign language recognition is generally divided into two major categories: Dataglove-based sign language recognition and vision-based sign language recognition. Vision-based sign language recognition attracts more attention of the researchers for it is more convenient for users than Dataglove-based sign language recognition. However, the state of art of vision-based sign language recognition is far from real application due to the difficulty of extracting efficient sign language features from video or image. One of great challenges in vision-based sign language recognition is to achieve view independent sign language recognition because view variance brings feature variance. Most of the current methods in vision-based sign language recognition require a specific view of the signers, generally the frontal view, so as to guarantee similar features between the training samples and the test samples. The constraint of a specific view means that the signers can only perform with specific location and orientation, and limits the freedom of the signer. The authors aim to achieve viewpoint independent sign language recognition within a certain scope with only one camera, so as to remove the restriction of the specific view and provide convenience for users. In the previous works, the authors based on the epiplor geometry and proposed the efficient method of verifying the uniqueness of fundamental matrices for viewpoint free sign language recognition. However, the method of verifying the uniqueness requires at least 8 point correspondences between one frame pair and will fail under data deficiency. So the authors propose the Sample-Consensus method in this paper for efficient viewpoint free sign language recognition under data deficiency. This research is sponsored by the National Natural Science Foundation of China under contract Nos.60533030, 60603023 and U0835005 and partially sponsored by open project of Beijing Multimedia and Intelligent Software Key laboratory in Beijing University of Technology. One purpose of these projects is to solve the key problems in sign language recognition.