When writing a survey, the goal is to make it as easy as possible for your respondents to give you accurate information. If your respondents are confused or frustrated, you won’t be able to collect information that you can act on. With more than 75 years combined experience in writing surveys, the Custom Research team has highlighted their top tips for writing an understandable questionnaire that will get you actionable data:

Ask only one question at a time

You want to get as much information as possible from your respondents, and it’s easy to be tempted to cram a lot into each question. Unfortunately, that makes it difficult to determine what the respondent is actually saying. If a restaurant’s questionnaire asks “Rate your server’s friendliness and efficiency on a scale of 1-10,” there’s no way of knowing if the respondent’s rating correlates with “friendliness” or “efficiency”. Asking two separate questions, “Rate your server’s friendliness…” and “Rate your server’s efficiency…” will give you a more accurate understanding of the traits you’re seeking to measure.

Use easy-to-understand language

It’s tempting to use technical jargon when writing questions- after all, this is your product or service that you’re writing about, so you have a deeper familiarity with it than a potential survey respondent. However, while this may be perfectly clear to you, respondents may be puzzled. When you write survey questions, it’s best to be as simple and clear as possible. If you’re not certain that your question is easily understandable, ask an outsider! Consider asking a friend or family member (or even a coworker unfamiliar with your product or service) the question to make sure they understand it.

Don’t lead the witness

Make sure you’re not asking questions that inadvertently suggest what sort of answer you want to receive from the respondent. For example, you should avoid phrases like “Most people say… Do you agree?” or “Wouldn’t you say that…” This type of phrasing encourages respondents to answer in certain ways, regardless of what their true opinion is.

Don’t give away the answer

Sometimes you’ll need to ask both open ended (respondents answer with their own thoughts) and closed ended (respondents react to specific ideas/statements you provide) questions about a topic. In those cases, make sure the open-ended questions are asked first.  For example, suppose you want to know what’s important to consumers when they’re shopping for your type of product. First, ask them the question and allow them to respond with whatever comes to mind (this would be the “open-ended” question). You could follow up with a closed-ended question asking them to select qualities that might be important to them from a list you provide. By asking the open-ended version of the question first, you’re ensuring that you’re getting qualities that are top-of-mind for the respondents. If you’d asked the closed-ended version first, they may just be repeating the qualities that you listed, without mentioning what’s truly important to them.

Make it easy for them to be honest

It’s important to keep your identity under wraps when conducting a survey. You want honest opinions on your products and services, so you need respondents to remain unbiased. If respondents know that this is your survey, one of two things could happen:

• Respondents may be “nicer” than they would otherwise be, so as not to seem mean OR
• Respondents may be even harsher than usual, as a way to “get back” at you over negative past experiences. (Not that this feedback isn’t important, it’s just more valuable when the respondent doesn’t have an ulterior motive)

In both cases, you’re not getting an accurate reflection of the respondent’s views. One way to camouflage your own involvement and get honest responses is to ask questions about your competition in addition to your own products. Not only do you get useful competitive information, it makes it harder for respondents to discern who created the survey (which is much easier for them to do if your product or service is the only one being asked about).

Try and keep your scales consistent

The types of questions you’re asking will generally help you determine what kind of scales to use. Depending on the questions, you will need to consider:

• Scale Polarity
  • bi-polar (have a middle/neutral response with positive and negative options on either side)
  • unipolar (responses are all positive to different degrees)
• Number of options
  • 5, 7, or 10 options are typical
  • Even number of options (forces a positive or negative response) vs. odd number of options (allows for a neutral response)

By keeping scales consistent (i.e.; a bi-polar scale with 5 options) as much as possible, you’ll make things easier for the respondent, which in turns means you’ll receive more accurate responses from them. When the scales are similar across the board, the respondent quickly gets familiar with the task, and can answer questions confidently. If each question has different scales, the respondent will constantly have to “relearn” the task, which is frustrating at best and confusing at worst—and could lead to less accurate responses.

Randomization and Rotation

As we mentioned with setting up the order of open-ended and closed-ended questions, the very order of questions in your survey can introduce bias. One of the ways to avoid that is to randomize and rotate the order that questions appear to different respondents. With a list of attributes, randomize the order for each respondent—though the list should appear in that same order for the respondent every time in order to minimize confusion. This way that over the course of getting all the completed surveys, each attribute will have opportunities to appear at the beginning, middle, and end of lists. If you’re evaluating multiple stimuli, such as two brands or concepts, rotate the order in which they appear in the survey. For example, half of the respondents may see Option A followed by Option B, while the other half see Option B followed by Option A.  (As a side note, don’t go back and forth between options with your questions.  Ask all your questions for Option A and then move onto Option B.   This allows respondents to keep their focus on a particular option.) Methods like randomization and rotation help eliminate the inherent bias that’s introduced if everything is always shown in the exact same order

Make sure your answers are mutually exclusive

If the same answer appears in two options in a question, it will confuse the respondent and result in  data errors. For example, suppose you want to know how much someone spends each week on groceries. The respondent wants to answer $50. If you use this scale:

• $0-$50
• $50-$100
• $100-$200
• More than $200

The respondent won’t be sure which option to check since each of that amount appears in 2 different places. By slightly adjusting your scale to:

• Less than $50
• $50-$100
• $101-$200
• More than $200

You’ve solved the problem and made it easier for the respondent to give you an accurate answer.

Ultimately, the quality of the data you collect will only be as good as the effectiveness of the survey you use. As a final tip, once you have written and programmed your survey be sure and test it yourself and with others. This will confirm it is working the way you envisioned and provide reassurance regarding the data you are gathering.

Need help writing your survey? The Stevenson Company’s experts can give you an unbiased view of your survey. Our experts have decades of experience writing, implementing, and analyzing surveys. Contact us today for your free consultation.