We are developing technologies to estimate the psychological states of TV viewers from their physiological and behavioral reactions in order to objectively analyze the psychological effects of programs on viewers. In 2009-2010, we developed an environment for measuring brain activity with a combination of functional near-infrared spectroscopy (fNIRS) and electroencephalography (EEG). We also measured the eye-gaze distributions of viewers (i.e., the part of the screen that the viewers is gazing at) by using an eye-gaze tracking system and analyzed the characteristics of video that is more likely to draw someone's attention. Furthermore, we analyzed the characteristics of video with shaky or flashing images that induce unpleasant feelings.

Since fNIRS requires less effort on the part of the test subjects and is resistant to electromagnetic noise, it is suitable for measuring the brain activity of someone watching video. However, it is impossible to estimate the anatomical locus of brain activity signals, and the temporal resolution is poor because the metabolic state in the cerebral cortex is used as a marker of brain activity. To deal with these problems, we developed an environment in which the signal source is estimated by using three-dimensional position measurement devices equipped with magnetic sensors, and which has high temporal resolution since it is possible to perform electroencephalography (EEG) measurements simultaneously with fNIRS.

To investigate which parts of TV programs attract viewers' attention and interest, we collected eye-gaze data from 80 individuals while they were watching video clips. These measurements were made using the eye-gaze tracking system that we developed in 2008-2009 to enable simultaneous eye-gaze measurements of up to five people. We verified that this measurement system can be used to collect data efficiently, and we analyzed the relationship between the viewer's eye-gaze distribution and the physical features of the clips. We measured the eye-gaze distribution of viewers while they were looking at short video clips (5 seconds) with no sound (see Figure) so as to suppress the effects of factors such as prior knowledge and the context of the clips, and we compared these results with the eye-gaze distributions estimated by a model based on visual information processing mechanisms. The results of this comparison suggested that it is possible to optimize the parameters of this model.

Figure. Example of a measured eye-gaze distribution

As the screens of home TVs become larger, more viewers may feel a sense of unpleasantness when shaky and/or flickering footage is shown. We have therefore started researching technology that detects unpleasant scenes automatically. In 2009-2010, we used a subjective evaluation method to analyze the relationship between the physical characteristics of moving images and unpleasant feelings, and we developed an algorithm for estimating the degree of unpleasantness from the physical characteristic quantities of each scene.