Doctor of Philosophy, The Ohio State University, 2009, Electrical and Computer Engineering

In American Sign Language (ASL), the manual and the non-manual components play crucial semantical and grammatical roles. The design of systems that can analyze and recognize ASL sentences requires the recovery of both these manual and non-manual components. Manual signs in ASL are constructed using three building blocks – handshape, motion, and place of articulation. Only when these three are successfully estimated, can a sign be uniquely identified. The first part of my research is to define algorithms to recognize manual signs based on the recovery of these three components from a single video sequence of two-dimensional images of a sign. The 3D handshape is obtained with a structure-from-motion algorithm based on the linear fitting of matrices with missing data. To recover the 3D motion of the hand, a robust algorithm is defined which selects the most stable solution from the pool of all the solutions given by the three point resection problem. Faces of the signers in the video sequence are detected, with which the coordinate system with respect to the signer is defined and hence we recover the place of articulation of the sign. Based on the recognition results of the three recovered components, the manual signs are recognized using a tree-like structure. For the non-manual component of ASL, we need to provide an accurate and detailed description of external and internal facial features. The second part of this research focuses on the precise detailed detection of faces and facial features. Learning to discriminate the features from their context permits a precise detection of facial components, which is the key point of the feature detection algorithm. And because the shape and texture of facial features vary widely under changing expression, pose and illumination, the detection of a feature versus the context is challenging. This problem is addressed with the use of subclass division, which is employed to automatically divide the training samples of each facial (open full item for complete abstract)

Committee: Aleix Martinez PhD (Advisor); Yuan F. Zheng PhD (Committee Member); Mikhail Belkin PhD (Committee Member) Subjects: Electrical Engineering