With all kinds of tools like typeform, surveymonkey, google forms at the tip of our hands, it seems that everyone can do market research. But I often see mistakes being made in “home-made” questionnaires, that will lead to the wrong outcomes, and consequently to the wrong business strategy.
- No clear research objectives: if you don’t know exactly what you want to do with the outcomes of your research, the questions will be all over the place. This is not only bad for the company you’re working for, but also for the respondent. Why do you ask me which brands I don’t use, but not which brand I do use?
- Incorrect research method: If you want to know someones shoe size, it’s easy to ask. It already gets more difficult with their weight. A common question is which version of three packaging designs will perform best (in unit sales). You don’t get the answer by showing your respondents the three different versions and ask which one they prefer. In the store environment, they will only see one. So, you need to test the purchase intention for each version within a comparable group of respondents (this is called monadic design)
- The questionnaire is too long/complex: Read why that’s a problem for many reasons in this blog.
- Asking sensitive questions too early in the questionnaire: The worst example is asking for the email address of the respondent in the first question after stating in the introduction that the questionnaire is anonymous.
- Not including demographic questions: Knowing the respondents’ age, gender, region, education, income etc helps to make sure your response is representative and it also helps you when segmenting the data. In the responses of questionnaires made by students, you often see an over representation of people in their twenties (their peers) and people in their fifties (their parents, who are very helpful in posting the survey on LinkedIn)
- Failing to ease the respondents into the questionnaire: A questionnaire that begins with very detailed questions without any introduction will make respondents feel disconnected from the survey. So first ask how often someone drinks tea, before asking very detailed questions about why they drink a specific kind of tea.
- Asking leading questions: “Don’t you agree that this brand X is superior in its use?” is an example of a leading question that assumes the respondent agrees with the statement. And it’s also confusing, because it has a negative aspect (don’t) in there, which can make it confusing for the respondent to answer. (“I agree that the product is the best, but do I answer with yes, or no then???”)
- Biased language in the questions or responses: “Why do you think our product is better than our competitors?” is an example of a question that assumes the product is superior to competitors and may bias the response.
- Using jargon instead of consumer language: “How often do you eat dry sauces?” is something a random respondent will not completely understand. You have to rephrase the question or explain that dry sauces are sauces made from a dry substance (e.g. a powder mix).
- An incomplete list for a single- or multiple choice question: Some time ago, I encountered a question on the restaurants I frequent. A fast food chain clearly had issued this questionnaire, as the answer list only contained fast food chain. The answers: “an independent restaurant”, and “other, …” clearly lacked in the list.
- Using (too many or difficult to answer) open-ended questions: Respondents don’t like to answer open-ended questions, so keep them to a minimum. Secondly, be very clear on what kind of answer you want. Just asking: “what do you think about brand X?” will give you responses like: “nice”, “expensive”. Thirdly, in students’ questionnaires I sometimes see only open-ended questions, and then I feel a pity for the students who have to analyse and read all of the responses.
- Wrong or no routing: Some questionnaire makers think everyone is as involved with their brand as they are. So they forget to ask about brand knowledge, or forget to let respondents skip questions that are not relevant to them (e.g. when they are a non-buyer).
- Failing to test the questionnaire: If you pretest the questionnaire before launching it (ideally with 1-2 people on the spot to hear their comments and e.g. online with 50 pre-respondents), you will be able to sharpen up some questions, answer lists and routing issues.
As easy as it may seem, making the perfect questionnaire is not a walk in the park. Spinos | Research & Analytics can help you with many market research questions for a limited budget. Please send an email with your research question, and I will get in touch asap.