Microsoft Is Teaching Kinect to Understand Sign Language

Microsoft researchers are working on ways to use the Kinect's body-reading capabilities to understand sign language inputs.

Microsoft Research Asia and the Chinese Academy of Sciences' Institute of Computing Technology has collaborated to use the Kinect to help with computer sign-language recognition. As shown in the video above, they've demonstrated the Microsoft Kinect's ability to translate signs by tracking hand and body actions.

Microsoft Research says in "translation mode" the Kinect can translate sign language into text or speech. Both isolated word recognition and continuous sentence recognition were tested. In "communications mode," an avatar can help an individual who is deaf or hard-of-hearing communicate with someone who can hear by converting the sign inputs into text and vice-versa.