September 2014
From an Experimental Survey on Face-to-Face Method
Akiko Hoshi
When NHK conducted a comparative study of surveys in 2008, where the eighth “Survey of Japanese Value Orientations” which was carried out by NHK itself and an experimental survey outsourced to an external data collection organization and implemented in a parallel manner as much as possible were compared, different response rates and different response distribution for certain questions were observed. The experimental survey had the following notable differences from the original survey: (1) lower response rate, (2) many nonresponses due to refusal, (3) fewer response alternatives in questions with multiple answers allowed, (4) respondents tendency to answer innocuously to presumably difficult-to-answer questions, and (5) fewer favorable responses to questions regarding TV and radio.
This article examines whether such differences in survey results depending on data collection organizations can be solved by operational contrivance in the implementation phase. In 2013, in parallel with the ninth “Survey of Japanese Value Orientations,” we conducted an experimental survey, which was carried out after we had examined what factors in the implementation stage were causing each difference and taken measures to prevent it. As a result, the response rate was improved, but other differences were not solved sufficiently enough.
In a comparative study of surveys conducted by different data collection organizations, it is necessary to take note of possible differences in the results when analyzing them.