Skip to Main Content
 

Global Search Box

 
 
 
 

Files

File List


osu1237564092.pdf (32.04 MB)

ETD Abstract Container

Abstract Header

Modelling and Recognition of Manuals and Non-manuals in American Sign Language

Ding, Liya
http://rave.ohiolink.edu/etdc/view?acc_num=osu1237564092

Abstract Details

2009, Doctor of Philosophy, Ohio State University, Electrical and Computer Engineering.
In American Sign Language (ASL), the manual and the non-manual components play crucial semantical and grammatical roles. The design of systems that can analyze and recognize ASL sentences requires the recovery of both these manual and non-manual components. Manual signs in ASL are constructed using three building blocks – handshape, motion, and place of articulation. Only when these three are successfully estimated, can a sign be uniquely identified. The first part of my research is to define algorithms to recognize manual signs based on the recovery of these three components from a single video sequence of two-dimensional images of a sign. The 3D handshape is obtained with a structure-from-motion algorithm based on the linear fitting of matrices with missing data. To recover the 3D motion of the hand, a robust algorithm is defined which selects the most stable solution from the pool of all the solutions given by the three point resection problem. Faces of the signers in the video sequence are detected, with which the coordinate system with respect to the signer is defined and hence we recover the place of articulation of the sign. Based on the recognition results of the three recovered components, the manual signs are recognized using a tree-like structure. For the non-manual component of ASL, we need to provide an accurate and detailed description of external and internal facial features. The second part of this research focuses on the precise detailed detection of faces and facial features. Learning to discriminate the features from their context permits a precise detection of facial components, which is the key point of the feature detection algorithm. And because the shape and texture of facial features vary widely under changing expression, pose and illumination, the detection of a feature versus the context is challenging. This problem is addressed with the use of subclass division, which is employed to automatically divide the training samples of each facial feature into a set of subclasses, each representing a distinct construction of the same facial component. This approach is combined with edge and color segmentation to provide an accurate and detailed detection of the shapes of the major facial features. This proposed detection algorithm is used to obtain precise descriptions of the facial features in video sequences of ASL sentences, where the variability in expressions can be extreme. With the proposed algorithms, the modelling and recognition of ASL manual signs using the three manual components are achieved, and the non-manuals of ASL are detailedly and precisely modelled, which provides data for the analysis and recognition of the non-manuals in ASL. The recognition of both the manual and the non-manual components enables human-computer-interface systems to understand ASL.
Aleix Martinez, PhD (Advisor)
Yuan F. Zheng, PhD (Committee Member)
Mikhail Belkin, PhD (Committee Member)
124 p.
Electrical Engineering
Computer Vision; Pattern Recognition; Manual Sign Recognition; Face Detection; Context Feature; Subclass Based Classification; American Sign Language

Recommended Citations

Citations

  • Ding, L. (2009). Modelling and Recognition of Manuals and Non-manuals in American Sign Language [Doctoral dissertation, Ohio State University]. OhioLINK Electronic Theses and Dissertations Center. http://rave.ohiolink.edu/etdc/view?acc_num=osu1237564092

    APA Style (7th edition)

  • Ding, Liya. Modelling and Recognition of Manuals and Non-manuals in American Sign Language. 2009. Ohio State University, Doctoral dissertation. OhioLINK Electronic Theses and Dissertations Center, http://rave.ohiolink.edu/etdc/view?acc_num=osu1237564092.

    MLA Style (8th edition)

  • Ding, Liya. "Modelling and Recognition of Manuals and Non-manuals in American Sign Language." Doctoral dissertation, Ohio State University, 2009. http://rave.ohiolink.edu/etdc/view?acc_num=osu1237564092

    Chicago Manual of Style (17th edition)

osu1237564092
451
© 2009, all rights reserved.
This open access ETD is published by The Ohio State University and OhioLINK.