October 2011
Analysis of How Respondents Fill in Questionnaires, And a Model Schedule for Mail Surveys
Noriko Onodera
Based on the 2008 and 2009 test mail surveys conducted in 2008 and 2010, the author discusses the methodology of mail surveys in three reports.
In this second report, the author analyzes how respondents filled in questionnaires in the 2010 mail survey. Grasping the reality of respondents’ fill-in behaviors is highly important because mail surveys give them more freedom in terms of answering questionnaires, which may influence the responses.
The author also examines what types of survey scheduling will be effective to collect questionnaires more promptly and gaining higher response rates by analyzing the differences in the schedules and collected statuses of the 2008 and 2010 surveys.
The analysis of fill-in behaviors revealed the following. (1) Respondents do not necessarily return the questionnaires immediately after they complete it, with some doing so just because of the reminder or deadline. (2) Some respondents do not understand the way to answer the questionnaire well; sometimes they consult with other family members and fill in their opinions, not the opinions of respondents themselves. (3) While some answer the questionnaire just casually, without thinking deeply, others take time to answer each question and therefore feel the burden of completing the questionnaire. (4) Although it is said one of the advantages of mail surveys is “easier to ask hard-to-ask questions,” which leads to “easier to know respondents’ true feelings,” some respondents do avoid answering such questions. In order for respondents to complete the survey in the proper way, it is indispensable to plan a survey schedule and prepare survey materials and questionnaires, taking these respondents’ fill-in behaviors into consideration.
The schedule of mail survey includes sending a request letter for cooperation prior to the actual survey, sending a questionnaire, and sending a reminder twice. The deadline is indicated both on the questionnaire and each reminder. Based on the analysis of the fill-in behaviors, the author suggests the following. (1) The deadline set in the questionnaire should be Monday and the questionnaire should be sent out so that the respondents have two Saturdays and Sundays before this deadline. (2) The deadline indicated in the reminder should not give too much leeway, but should be at least a week after the reminder is sent out. (3) Saturday and Sunday should be included in the period between the arrival of a reminder and the deadline because many respondents fill in questionnaires on weekends. (4) Reminders should be sent out promptly, not taking too many days after the deadline indicated in the previous mail. Sending two reminders is necessary to achieve higher response rates, but receiving mails many times may bustle the respondents. Therefore, it is important to seek understanding of the respondents about the survey so that they spontaneously become willing to cooperate with the survey.