Bad Survey Questions

Leading questions are phrased in a way to drive the users’ responses towards a specific option. A survey uses a leading question if the researcher wants the respondents to answer a certain way. The problem is that you’ll never really know the honest opinions of users. 

Examples of leading questions – 

• How satisfied are you with the product?

Fixed: On a scale of 1-10, how would you rate our product?

• What problems did you face when using our website?

Fixed: What was your impression of our website?

• Our restaurant has been given a 5-star rating by our customers. How would you rate your experience?

Fixed: Removing the first sentence is a better way to ask the question.

Asking negative questions 

A negative question is a survey question in which a “yes” signifies a negative response and “no” signifies a positive response. Such kinds of questions have tricky wordings that may confuse the respondents. They may end up choosing the wrong option as their response which will affect the survey result. 

Examples – 

• I don’t know about Machine Learning.
  • Yes
  • No
• I didn’t enjoy visiting your restaurant.
  • True
  • False

Reframe the questions as given below: 

• How well do you know about Machine Learning?
  • Not at all
  • Fairly well
  • Very well
  • Excellently well

• How will you rate your experience at our restaurant?
  • Excellent
  • Good
  • Average
  • Poor

Questions without a “Not Applicable” option

If respondents are required to answer all questions in a survey, they may abandon the survey. There may be situations when the respondent does not know the answer to a question or wish to skip a particular question. 

For example, 

• What is your annual income?
  • Less that $15,000
  • $15,000 to $36,999
  • $35,000 to $49,999
  • $50,000 or above

The above question that asks about the income of the respondents does not give them a “Not applicable” option. Not everyone is comfortable sharing details on how much they earn. Therefore, keep such questions optional or include a default entry.

Avoid double-barreled questions

Double-barreled questions are a very common example of a bad survey question. It aims to cover multiple subjects in a single question. The best way to identify such questions is to look for questions that have “and” or “or” in them. For example,

• How satisfied are you with our product and delivery?

This question forces the respondent to give an opinion on both the product and delivery. 

The question can be fixed by splitting it into two as follows – 

• How would you rate our product?
• How satisfied are you with the delivery of your product?

A good survey aims to ask only one question at a time to be able to gather valuable data. 

Using absolutes in a question

An absolute question restricts the response options to a simple “yes” or “no”. Always include a variety of options that the respondents can choose from. This will not only let them give honest answers but will also let businesses gain deeper insights.

For example,

• Do you always watch TV?
  • Yes
  • No

In the case of the above question, a respondent is most likely to answer with a “no”.

A better way to frame the question is as given below,

• How often do you watch TV?
  • 2 hours per week
  • 4 hours per week
  • 6 hours per week
  • Above 6 hours

A few more pointers to take care to avoid a bad survey are-

1. Understand the audience of the survey before listing down the questions. Learn about the demographics, their language preferences, and their needs. 
2. Ensure the number of questions is neither too high nor too low. Include a variety of questions for reliable results.
3. Ask short, simple, and precise questions. Avoid using phrases or slang that are complicated.
4. Make use of both open-ended and closed-ended questions. Focusing more on open-ended questions will consume a significant amount of the respondents’ time.