Since Tokyo was selected as the host city of the 2020 Olympic and Paralympic Games interest in sports has been further rising in Japan. Broadcasts and internet delivery of sports content will be an effective tool to convey the attraction of the Olympics and Paralympics to a large number of users and resonate with them. However there also are some problems to solve to make content more easy-to-understand so that more people will be able to embrace sports coverage.
To make sports broadcasts more easy-to-understand and attractive, the NHK Science & Technology Research Laboratories has been pursuing research and development of new visual expressions and “audio description” –a new approach to sports commentary that will enable persons with a visual impairment enjoy liver broadcasts of sports.
This paper first reports the latest research activities with examples of video production: object tracker that follows the movement of balls and other objects with high accuracy and stability to superimpose the trajectory, multi-viewpoint robot camera system that generates dynamic images with views smoothly rolling around the moving subject for crucial moments such as a goal, 2.5D multi-motion representation technique that shows a series of motion of athletes in the form of a serial photograph, and a method to estimate the head pose of a soccer player that would help commentary of the games. To make the live coverage of the content produced with these technologies more accessible to and more enjoyable for the visually impaired, we are developing “automatic audio description” of running commentaries, which is also presented in this paper.