Series: Video Bank: A New Video Management System
Camera Parameter Data Acquisition Technology
Hideki Mitsumine
Integrated Broadcast-Broadband Systems Research Division
The “Video Bank” is a system that automatically analyzes sensor data acquired when shooting and storing video materials in an effort to find supporting information (metadata) for making indexes for video retrieval and processing during program production. In this article, I will discuss its camera-parameter data-acquisition technology.
Recent program productions have frequently employed video processing to enable computer graphics (CG)-rendered subjects to be incorporated in video segments shot with a camera. This technology renders CG images corresponding to camera movements during the video processing, and it makes for more natural video compositions of CG subjects in the actual shots. The conventional ways of estimating camera movements (or camera positions) for video processing use either sensor technology or video analysis technology. The former method is robust but has lower accuracy, and it requires the camera to be mounted on a special tripod or crane. In contrast, the latter can be very accurate, but its accuracy is highly dependent on the content of the shot. These weaknesses have been alleviated and the advantages of the two techniques have been consolidated in our new sensor-integrated technology.
Hybrid sensor
The camera movement restriction was overcome by using a small “hybrid sensor”, composed of a gyro sensor and sensor camera facing the floor, to measure the camera direction and travel distance along the floor.
Video analysis technology controls feature point miscorre-spondence-based reductions in accuracy
While video analysis estimates the camera’s position by extracting and tracking a certain characteristic component in the actual shot subject image (natural feature point), some objects cause feature point tracking errors that degrade the estimation accuracy. We suppressed the tracking error by using data indicating the relative positions of stationary objects, such as studio set, props or furniture, in the image.
Sensor-integrated technology
The system estimates the camera parameters’ accuracy during the video analysis and re-estimates the camera’s position based on the camera parameters of the hybrid sensors only in the lower-accuracy video segments. This enables automatic acquisition of highly accurate and robust camera-parameter data.
With the goal of applying these technologies to program production, we will continue our research on high-speed video analysis and interfaces using a diverse range of video processing tools.
