August 2011
Definition of Response Rate for Mail Surveys
Noriko Onodera
As advantages of mail surveys include high response rates and lower costs, it is highly probable that NHK will employ this survey method in the future. Therefore, we are now working on the standardization of methodology. Based on two test mail surveys conducted in 2008 and 2010, the author reports what has been discussed so far in three installations. The August issue presents the outline of the 2008 and 2010 test mail surveys. The author also analyses “responded by proxy,” “filled-in by proxy” and “consultation” that tend to occur in mail surveys where respondents themselves are asked to complete the questionnaire, in order to confirm the definition of “response rate.”
In case of mail surveys, where no polltakers involve, returned questionnaires sometimes include answers which can be regards as improper such as someone but the eligible respondent entering his/her own opinion or a family member filling in the respondent’s opinion by proxy. Given such possibilities, the author examines how “response rate” should be defined and how the eligibility of the “response” can be confirmed?
The 2010 survey saw such cases of “responded by proxy” where people other than respondents entered their opinions (50 out of 1,800 people surveyed, 2.8%), “filled-in by proxy” where family member filled in respondent’s opinions on behalf of them (48 respondents, 2.7%) and “consultation” where respondents consulted with family members or other people regarding the answer (78 respondents, 4.3%).
As for “responded by proxy,” it is “impossible” to regard them as eligible because they are not the opinions of respondents but of other people. However, in case of “filled-in by proxy,” where actual respondents’ answers are entered by other people, these answers should be considered as eligible “response” because otherwise certain respondents who are not able to complete the questionnaire due to eyesight or other problems would have to be excluded from the population.
It is highly difficult to confirm whether a returned questionnaire is “responded by proxy” or not. In general, if the gender or age on the questionnaire is different from those on the sample list, the surveyor would determine whether “the respondent answered the right gender and age,” and therefore how to set the criteria is important.
At all events, to eliminate cases of “responded by proxy,” “filled-in by proxy” and “consultation” as many as possible, it is also important to make various efforts when designing questionnaires and accompanying letters.