Today I’ll focus on writing good questions. While this may seem like the main substance of survey creation, figuring out what and why you’re researching in the first place, from Part 1, should come first.
1. Write questions in a way that reduces bias.
Asking a question like “Do you enjoy using our services?” tends to lead to acquiescence bias: people are generally agreeable, so they’ll give more positive responses because you phrased the question in a positive way. But that’s not going to give you the actionable data you’re looking for.
Instead, you could do a little formatting and rephrase it this way:
“Please select the option that best matches how you feel [with options displayed in a straight line]:
(1) I don’t enjoy using these services (2) (3) I feel neutral about these services (4) (5) I enjoy using these services.”
Putting all options out there makes people feel like it’s ok to give a neutral or negative response, and if you’re surveying for the right reasons, you’ll want that information to help identify areas for improvement.
Little things like this can make a big difference in the quality of responses you get.
2. Avoid leading questions.
Another form of bias is giving people information that might influence their opinions right before asking them a question. This happens in fundraising and political polling all the time.
Instead of:
“Last year our employee group hosted 12 professional development events. How satisfied are you with our employee group’s work in the last year?”
Just leave out the first part or ask their opinion before presenting this information:
“How satisfied are you with our employee group’s work in the last year?”
If you really want to get useful information from your survey, then don’t try to trick respondents into answering a certain way.
3. Be specific.
People interpret vague questions in different ways, making the results less meaningful.
Consider how you might interpret the question:
“How often do you accomplish your errands by walking or using a wheelchair?”
– Frequently
– Sometimes
– Rarely
– Never
You might interpret “frequently” as “every day,” or you might interpret it as “every time I run an errand,” which might only be once a week. Others might think that once a month counts as frequently, compared to how often they walk otherwise.
These types of options can be useful in certain contexts, but generally speaking it will make it hard to interpret your survey results. Use more specific language such as “every time I do an errand” or “about half the time I do an errand” instead if you want to be sure that everyone is interpreting the answer choices similarly. Or use “about once a week” or “2-4 times per week” if you want comparable results regardless of how often the respondent runs errands at all.
Next week: How to ensure you hear from the right people.
Nerd on the Bus 