SMILE: Automated sign language recognition for sign language tests

Category Project

Ausgangslage und Ziele

SMILE (Scalable Multimodal Sign Language Technology for Sign Language Learning and Assessment) is a project set up to develop a sign language test system that draws on automated sign language recognition.

Project Management

Sarah Ebling Title Dr.

Facts

  • Duration
    03.2016
    02.2020
  • Project number
    4_32

Project Team

Methodical approach

«p>The project adopts a multidisciplinary approach: research strands devoted to sign language technology and sign language testing are combined with sign language linguistics.

The project consortium consists of three institutions optimally placed to complement one another with their skills:

  1. The Idiap Research Institute based in Martigny (CH) will coordinate the project. The Institute will also develop an innovative approach to automated sign language testing and feedback, inspired by an approach to language recognition developed for the SNF project »FlexASR”.
  2. The University of Applied Sciences of Special Needs Education (HfH) based in Zurich (CH) is contributing its expertise in the field of sign language testing and sign language linguistics (in conjunction with the Centre for Sign Language Research in Basel). The HfH will function as the interface with L2/foreign language learners and the deaf community in German-speaking Switzerland.
  3. The University of Surrey (GB) has for many years held expertise in the field of sign language technology, visual data acquisition and computer vision. As a partner in the (now completed) EU project »DictaSign”, the University of Surrey continues to benefit from broad knowledge of current research on sign language technology.

Results

The SMILE project is founded on the Common European Framework of Reference for Language (CEFR). It is developing a test system to measure the production of Swiss-German Sign Language (DSGS) vocabulary at Level A1, drawing for the first time on new sign language technologies. It should result in the development of a platform for DSGS-specific teaching and learning systems which can serve as a model for other sign languages.

Publications