Color Moiré Reduction Technology for the Integral 3D Display

2.1 Causes of color moiré patterns

In this section, we describe the mechanism behind the appearance of color moiré patterns, referring to the schematic diagram shown in Fig. 1.

As noted above, direct-view display panels typically possess R, G, and B subpixel structures. For example, in the case of vertically striped subpixels, the R, G, and B subpixels are arranged in horizontally repeating patterns across the screen, and sampling by a lens array of the type shown in Fig. 1 thus gives rise to distortion due to low-frequency aliasing.

Among each set of subpixels, let us focus on the particular case of the G subpixels. The pixel pitch Ppx and its spatial frequency f_px are related as P_px = 1/f_px. By denoting the elemental lens pitch by P_Lx = 1/f_Lx and any integers by n, a spatial frequency f_m at which moiré patterns are produced by pixels and elemental lenses is given by

Figure 2 shows Equation (1) on the frequency axis; here, solid black lines denote carrier frequencies due to elemental lenses, while red dashed lines denote image frequencies due to pixels. As this figure demonstrates, moiré patterns at frequencies below the elemental-lens sampling frequency f_Lx appear prominently in three-dimensional images. This phenomenon arises in identical fashion for each of the R, G, and B subpixel arrays, and the resulting moiré patterns for each color appear superposed atop one another with relative phase shifts of 1/(3f_m), producing the effect seen by observers as color moiré patterns. The mechanism responsible for color moiré patterns is thus the same as that for monochromatic moiré patterns; in the following discussion, we will often refer only to the specific case of G (green) moiré patterns, as the R and B cases are understood to be equivalent.

Next, we consider the case in which the entirety of a single display panel displays green with 100% brightness. If subpixels are arranged in a vertical-stripe configuration, we have periodic components only in the horizontal direction, while in the vertical direction, only direct current (DC) components are present; thus, it suffices to consider a one-dimensional problem involving only the horizontal direction (x-direction). In Fig. 3(a), we have extracted only the G subpixels from the display-panel substructure of Fig. 1; for simplicity, we take the pixel aperture ratio^*1 to be 100%, neglect black-matrix regions^*2, and assume that the RGB subpixel apertures are 1/3 of the full pixel aperture a_px = P_px. By taking into account the effect of defocusing during readout due to individual elemental lenses within the lens array, the display-panel brightness distribution c₁(x) is known to be a superposition of fundamental and higher harmonics expressed by a Fourier series of the following form:⁵⁾

Here, F(2πn) is the amplitude coefficient of the nth higher harmonics, J_k(·) is a kth Bessel function of the first kind, and ρ is the radius of the circle of confusion^*3 in the elemental image owing to elemental-lens defocusing. Also, sinc(n) = sin(πn)/(πn) is the normalized sinc function. By denoting the elemental-lens diameter by d, the focal length by l_f, the distance between the elemental image and the elemental-lens focus by g, and the distance from the lens array to the observer by L, the radius ρ may be expressed in the following form:

Then, the moiré modulation degree m(c₁) is given by

2.2 Moiré pattern reduction via synthesis of multiple display systems

We next discuss our technique for reducing moiré patterns by combining multiple display units. Fig. 4 and 5 respectively show schematic diagrams illustrating two-unit and three-unit synthesis configurations, while Fig. 6 shows phase diagrams that illustrate, for the three-unit case, the basic principle of our approach: by applying phase shifts of 1/3 of the phase to each of the color-specific moiré patterns before they are superposed, we achieve destructive interference that reduces the overall moiré patterns in the final superposition. We denote by c₂(x) and c₃(x) the brightness distributions appearing on the virtually synthesized display panel for the cases of two-unit and three-unit synthesis configurations, respectively. For the configurations shown in Fig. 4 and 5, we consider the problem of optimizing the relative positioning of the display units to achieve the maximal cancellation of moiré patterns; by considering this in terms of the relative positioning of the display-panel brightness distributions, it is clear that—as shown in Fig. 3(b) and 3(c)—the optimal procedure is to apply offset of 1/2 pixel (two-unit case) or 1/3 pixel (three-unit case) before superposing. Using Equation (2), we may express these conclusions in the form:

The moiré modulation degree is equal to the modulation degree computed from the brightness distributions appearing on these virtual synthesis display panels. Figure 7 shows plots of the results of computations using the modulation degrees m(c₁), m(c₂), and m(c₃) determined with Equation (5) for the moiré modulation degree. Note that the circle-of-confusion radius ρ in Equation (4) is negative if the elemental lens-to-display panel distance g is less than the focal length l_f, and positive otherwise.

From Fig. 7, we see that—excluding cases involving an extremely small circle of confusion due to defocusing (radius 0.03 pixels or below)—the two-unit synthesized display appreciably reduces moiré patterns compared with the single-unit display. As for the depth of moiré modulation, practical considerations suggest that, for the case of ρ = 0 in a single-unit display, a reasonable target is suppression at the level of 1% or below⁵⁾. In Fig. 7, the circle-of-confusion radius needed to ensure a moiré modulation depth of 1% or below is 1.62 pixels for a single-unit display, but only 0.56 pixels for a two-unit synthesized display. This reduction in circle-of-confusion radius—equivalent to a reduction in the extent of defocusing—has the effect of improving the modulation transfer function (MTF)^*4 for reconstructed images at positions far from the lens-array plane⁶⁾. Meanwhile, for the case of three-unit synthesis, it is theoretically possible to achieve complete moiré pattern suppression for any circle-of-confusion radius.

4.1 Moiré pattern reduction performance

Figure 9 is a photograph of the three-unit synthesis system that we assembled on the basis of the structure shown in Fig. 5. In this system, we first use one individual display unit to show a pure-white image—in place of elemental images—with the other two display units showing nothing (pure black); we capture images of the resulting display using a digital still camera for measurement purpose (Fig. 10(a)). Next, we show pure-white images on two of the three display units—with pure black on the remaining unit—then adjust the lens-array positions to minimize moiré patterns and again capture still-camera images of the resulting display (Fig. 10)(b). Finally, we show pure-white images on all three display units, adjust the lens-arrays position to minimize moiré patterns, and again capture still-camera images of the resulting display (Fig. 10)(c). For lens-array-position adjustments, we used a micro-actuating stage to move lens arrays over the range of a single pixel while actually observing moiré contrast (visibility) to determine the offset that minimized the moiré contrast. To ensure linearity of image data with respect to display-plane brightness, we set the image-acquisition gamma to 1. The image data acquired in this way was then analyzed to measure moiré-component contrast for each color. For contrast measurements, discrete Fourier transforms (DFTs) were applied to 20-lens-length horizontal one-dimensional waveforms of additively averaged contrast values for 12 vertical elemental lenses to determine modulation degrees for moiré frequency components. The numerical normalization of modulation degrees was defined by setting the DC component for each color (R, G, and B) to 1.

As an example, Fig. 11 shows the average brightness profile for B (blue) in the contrast-measured region. In this figure, the horizontal axis indicates the horizontal position x normalized by the elemental lens pitch P_L, while the vertical axis indicates brightness values normalized such that the DC component of the brightness value for the three-unit-synthesis display case is equal to 1; the blue dashed curve, red dash-dotted curve, and green solid curve respectively indicate results for the cases of the single-unit display, the two-unit-synthesis display, and the three-unit-synthesis display. With normalization chosen such that the moiré contrast for the single-unit display is 100%, for the two-unit-synthesis display, we find an average moiré contrast of 22% (R: 24%, G: 19%, B: 22%), while for the three-unit-synthesis display, we find an average moiré contrast of 10% (R: 9%, G: 9%, B: 12%).

Assuming a color space compliant with ITU-R recommendation BT.709¹⁰⁾, Fig. 12 shows CIE 1931 xy chromaticity diagram¹¹⁾ of RGB values over the regions covered by the average-brightness profiles discussed above. Chromaticity ranges for the cases of single-unit, two-unit, and three-unit display are indicated respectively in black, dark grey, and light grey. As the image used in our measurements is white everywhere, ideally, we expect the chromaticity to converge to the single central point corresponding to white, indicated by a circle in Fig. 12. From this figure, we see that, as the number of synthesized units increases, the chromaticity approaches white and the modulation degree of color moiré pattern decreases.

4.2 Resolution enhancement

For the single-unit display system and the three-unit-synthesis display system in the configuration shown in Fig. 5, we measured the resolution by displaying a resolution chart at positions of depth z = 0 mm (on the lens array) and z = 80 mm (in front of the lens array); the results are shown in Fig. 13. We see that the three-unit-synthesis display improves the aliasing distortion visible in the single-unit display within the central region of high spatial frequency. For z = 0 mm, the range of displayable spatial frequencies in the vertical direction is expanded; in the horizontal direction, there is no expansion in the displayable band—as expected on the basis of the theoretical analysis discussed above—but this band is still broad compared with what is observed in the vertical direction. In diagonal directions as well, we see angle-dependent improvements with good overall balance. For z = 80 mm, the region in which aliasing distortion had appeared is now replaced by DC components, reducing the extent of image-quality degradation. The jaggedness (sawtooth-like steps) discernible in the diagonal black/white boundary regions of the chart is also smoothed. These results are also in agreement with theoretical predictions.

Note that, when three display systems of equal brightness are synthesized, the resulting image is three times brighter than in the single-unit display case. For this reason, although the acquired images differ in brightness, in Fig. 13, we present images adjusted so that the average brightness in the single-unit display case matches that for the three-unit-synthesis case; this prevents brightness discrepancies from affecting the characterization of the moiré pattern or the resolution.

4.3 Combining multiple-unit synthesis with weak defocusing for moiré pattern reduction

In this section, in an effort to reduce the moiré modulation degree to levels below 1%, we induce weak defocusing by shifting the parameter g—the distance between lens arrays and elemental images—away from its focused value of 1.7 mm, and then attempt on-the-spot three-dimensional imaging with a circle-of-confusion radius of ρ≈−0.6 pixel. It is known that such defocusing conditions do not degrade reproducibility in the depth direction⁶⁾. As discussed in Section 2.2, increasing ρ by defocusing has the effect of reducing the moiré modulation degree for single-unit display. It is expected that even weak defocusing can reduce the moiré modulation degree to below 1% by combining it with multiple-unit synthesis. Figures 14(a) and (d) and Fig. 14(b), (c), and (e) show the experimental results obtained with the single-unit display and the three-unit synthesis display, respectively. Figures 14(d) and (e) show enlargements of subregions of Fig. 14(a) and (b), respectively. Figure 15 shows the normalized average brightness profile for the blue background region, measured under the same conditions as in Fig. 11. As the moiré modulation degree was greater for blue than for the other colors in the three-unit synthesis experiments described in Section 4.1, we focus here on the blue channel as a representative case. For the single-unit display case, the moiré modulation degree is approximately 2%, a level detectable by the naked eye. On the other hand, for the three-unit-synthesis display case, the moiré modulation degree is reduced to approximately 0.2%, a level essentially indiscernible by visual inspection. These moiré modulation-degree results were obtained via the DFT procedure described in Section 4.1.

Considering the overall resolution in Fig. 14, we observe appreciable improvements in the depiction of the character's eyes and head, as well as in outline profiles. The reduction in the extent of defocusing achieved by synthesizing multiple units yields increased resolution near the lens-array plane while simultaneously reducing the modulation degree of moiré patterns and retaining good depth-direction reproducibility performance for reconstructed images. These results demonstrate that the method proposed here can yield across-the-board improvements in the quality of reconstructed images.