ISSN 1000-1239 CN 11-1777/TP

### Tolerance Feature Extension of Substandard Sign Language Recognition with Finite Samples

Kong Leyi1,2, Zhang Jinyi1,2, Lou Liangliang3

1. 1(Key laboratory of Specialty Fiber Optics and Optical Access Networks(Shanghai University), Shanghai 200444);2(Joint International Research Laboratory of Specialty Fiber Optics and Advanced Communication(Shanghai University),Shanghai 200444);3(Key Laboratory of Wireless Sensor Network & Communication,Shanghai Institute of Microsystem and Information Technology,Chinese Academy of Sciences, Shanghai 200050)
• Online:2022-03-07
• Supported by:
This work was supported by the Subject Innovation and Talent Introduction Program (111) in Colleges and Universities (D20031) and the Key Disciplines Funded by Shanghai Education Commission(J50104).

Abstract: The expression of specious sign language in life is ambiguous, and the semantics of substandard gesture actions are easy to be confused. At the same time, it is difficult to obtain sufficient features for training sign language recognition model with finite samples, and the model is easy to over fit when it is too complex, which leads to low recognition accuracy. In order to solve this problem, we propose a representation learning method to expand the tolerant features of sub-standard sign language recognition with finite samples. This method based on the skeleton information of human body, facing the spatiotemporal correlation of sign language, constructes a autoencoder to extract standard features from a small number of original samples in sign language corpus; a large number of substandard samples are generated from standard features by generative adversarial networks, and then fault-tolerant features are extended by autoencoder to construct new features for subsequent sign language recognition tasks. The experimental results show that, under the condition of limited samples, the semantics of the samples generated by this method are clear, and the features of different semantics in the new feature set are easy to be divided. Using this method to build tolerant feature set in CSL dataset, the training sign language recognition model achieves 97.5% recognition accuracy, which indicates that it has broad application prospects.

CLC Number: