Chinese Journal of Computers   Full Text
Title	Establishment, Realization and Effectiveness Verification for Phonological Representation of Chinese Sign Language
Authors	NI Xun-Bo ZHAO De-Bin JIANG Feng CHENG Dan-Song
Address	(School of Computer Science, Harbin Institute of Technology, Harbin1 50001)
Year	2009
Issue	No.12(2438—2453)
Abstract &   Background	Abstract Sign language is a language. Linguists summarized grammar for sign language from the view of linguistics, and the theory of phonological representation was formed. Not having a system of sign digit in Chinese sign language, this paper firstly proposes a phonological representation according to the characteristic of Chinese sign language and language habit. Secondly, a method which combined data template of hand shape and Latin spectrum is proposed according to the characteristic of recognition system of Chinese sign language which is based on data glove, and the recognition system above is realized by the computer by means of automatic label. Finally, recognition system of Chinese sign language of data glove is lack of transcendental information, thus a confused word assemble with transcendental semantic information is established through the system of phonological representation. During the process of setting up the confused word assemble with transcendental semantic information, the semantic recognition function of the system of phonological representation is illustrated and analyzed. The confused word assemble based on statistical identification is compared with the one based on the prior sign-language semantic information. Without any change of statistic model, the experiment shows that the recognition function of phonological representation will to some extent improve the recognition rate which used the HMM，SOFM/HMM and Effort. Keywords phonology noting; labannotation; semantic meaning information; confusable vocabulary set; HMM; SOFM/HMM; effort Background Phonetic analysis of sign language as a kind of vision-sign language, which is completely different with sound language style, should be like that of sound language? The first trial of structural description to basic vocabulary unit of sign language was done by Stokoe. Before Stokoe, people believed that analysis to hand gestures was impossible which, as a whole, had only iconicity without internal structure. Stokoe firstly used the linguistics in sign language research to describe natural linguistic characteristics of sign language. Stokoe called the research on sign language structure the chorology. He thought the sign language consisted of place of articulation, hand shape and movement totally three factors. The structural analysis to sign language made by him laid a foundation for sign language research later on. The sign language linguistics afterwards made “grammar”-like summarization to sign language and hence the theory of phonological representation of sign language was formed. However, these theories of phonological representation of sign language are not realized actually in computer, which are mostly just theoretical description or noted by hand (in the form of video). Besides, these theories of phonological representation of sign language are not able to describe details of sign language movements. Basing on the situation that there is no notation system for Chinese sign language symbols, firstly, the paper proposes a phonological representation system according to the characteristic of Chinese sign language and language habit; secondly, a method which combined data template of hand shape and Laban spectrum is proposed according to the characteristic of recognition system of Chinese sign language which is based on data glove, and the recognition system above is realized by the computer by means of automatic label; finally, recognition system of Chinese sign language of data glove is lack of transcendental information, thus a confused word assemble with transcendental semantic information is established through the system of phonological representation, which is recognized with HMM, SOFM/HMM and Effort in the confused word assemble construed by the authors. Experiment results show that the recognition performance of the confused word assemble basing on statistic recognition comparing with that of the confused word assemble basing on transcendental semantic information, without changing statistic model, the method proposed by the authors helps HMM，SOFM/HMM and Effort to improve recognition rate to different extent. A confused word assemble with transcendental semantic information is established through the system of Chinese sign language phonological representation helps the statistic model or method to a certain extent to improve recognition rate, however, there are still a lot of recognition failures. The ways below should be used to solve the problem: ① make improvements to statistic models or methods such as HMM or SOFM/HMM; ② train model factors with more samples to improve recognition performance of statistic models or methods. The research work in the paper has been supported by Chinese National Natural Science Foundation Key Project(60533030)and Chinese National Natural Science Foundation Project(60603023). The paper is a bold attempt in the combination of sign language linguistics, kinesiology and computer science, which realizes effectively the phonological representation of sign language linguistics in computer and is meaningful to a certain degree as a reference for sign language linguistics and the recognition field of computer science. Meanwhile, the paper is theoretical basis and practical assurance for effectiveness verification to signer-independent sign language data.