Smart Production

Sign Language Synthesis Systems for Sports Information

To offer programs that the hearing impaired can enjoy better

Outline

We are researching on technologies that will make it easier for the hearing impaired to enjoy sports programs with sign language computer graphics (CG) generated using game data. We are presenting a system that automatically synthesizes sign language CG from data delivered during a game and a system that can efficiently produce sign language CG for sports news after the game.

Two systems developed for producing sign language CG

Features

Automatic sign language CG synthesis using game data

We developed a data-driven sign language synthesis system that analyzes sports game data and synthesizes sign language CG that shows the game status and progress in real time. We conducted trial automatic sign language CG synthesis using sign language text templates*1 for ice hockey and curling matches.

Translation assistance by sign language CG for sports programs

We developed a translation assistant system by sign language CG that enables synthesis of sign language CG from text outlines of sports news using corpus-based machine translations*2 and correction of sign language expressions with simple operations. We aim to not just display the machine translated results as is, but also to substitute sign language expressions as intended by the program producer.

Pictures

Translation assistance by sign language CG for sports programs
Automatic sign language CG synthesis using game data

Movie

Note

  • This research is partly being conducted in collaboration with Kogakuin University.
  1. Sign language template with numerical fields and proper nouns converted into variables, which can be filled in with the delivered game data
  2. Translation technology using machine learning of database collections of Japanese and sign language translated text samples