We launched new research on augmented reality (AR) and virtual reality (VR) and studied the concept of services that will offer new user experiences to viewers by using AR and VR technologies. In FY 2018, we researched “By AR/VR” services, which provide new viewing experiences by combining existing technologies, and “For AR/VR” services, which provide new viewing experiences by implementing technologies that have yet to be introduced in the AR/VR field and newly developed technologies.

■"By AR/VR" services

  We investigated how 2D images should be presented in a 3D virtual space of AR and VR. We organized the styles of using 2D images, the size of presented images and coordinate systems for the use of AR glasses. We divided the coordinate systems into six categories of (a) world coordinate system, (b) ambient environment coordinate system, (c) object coordinate system, (d) wrist coordinate system, (e) head coordinate system and (f) body coordinate system (Figure 1-10), and proposed appropriate coordinate systems for each style of use (Table 1-1).
  Focusing on 360-degree spherical images with an alpha (opacity) channel, we developed a method for cutting out an object to be presented from a 2D image and presenting it framelessly in a 3D space. We also devised a method that presents the images of an object with parallax by changing the radius of 360-degree spherical images with an alpha channel according to the distance between the camera and the object (Figure 1-11).

■"For AR/VR" services

  We investigated services that utilize high-resolution VR images to provide a new viewing experience. In FY 2018, we produced high-resolution VR images necessary for our investigation by combining (“stitching”) images captured with an array of multiple 8K broadcast cameras equipped with a fisheye lens. This approach succeeded in producing highly immersive images that can be used for head-mounted displays and other displays. We conducted capture experiments in multiple scenes using several production methods with different shooting distances and camera arrangements. For the shooting distance, we captured images at different distances from short to long. For the camera arrangement, we tried three different methods. We tested and investigated a method in which two 8K cameras are arranged facing opposite directions to capture 360-degree images and a method in which two 8K cameras are arranged side by side to obtain 3D VR images having binocular parallax in a forward direction. We also devised a new method for capturing VR images covering a viewing angle of 180 degrees or more by arranging three 8K cameras in the forward horizontal direction at different angles to increase the resolution to the maximum and allow flexible operation of production effects during program production. This method produced VR images that exceed 8K (Figure 1-12). We plan to examine the concept of services using high-resolution VR images by applying these methods.

 

[References]
(1)	Kawakita, Yoshino, Koide, and Hisatomi: "Ideal form of 2D moving picture in 3D space," HCG Symposium 2018, HCG2018-I-1-1 (2018)
(2)	Koide, Yoshino, Kawakita, and Hisatomi: "A study on high-immersive sense moving picture production for a high-definition virtual reality system using 8K-resolution cameras," ITE Technical Report, Vol.43, No.10, BCT2019-47, pp.45-48 (2019)