What Is A Biased Question

While customer surveys can yield amazing insights into what your customers want and need, they can also be a liability if the underlying survey questionnaire is flawed. 1 of the almost common causes of unreliable survey feedback is the biased survey question.

Customer feedback is not like shooting fish in a barrel to come by, which makes every survey response all the more meaning. It's important to make sure the input y'all collect from respondents is every bit unbiased as possible, and provides a clear lens into the customer feel. When you lot take bad survey questions in your questionnaire, you stop upwardly wasting a valuable opportunity to surface critical insights from customers and employees about how to improve your products and services.

What are biased survey questions?

A survey question is biased if it is phrased or formatted in a way that skews people towards a certain respond. Survey question bias besides occurs if your questions are hard to understand, making information technology difficult for customers to answer honestly.

Either style, poorly crafted survey questionnaires are a problem as they result in unreliable feedback and a missed opportunity to empathise the customer experience.

In previous posts, we've given examples of customer satisfaction questions and survey design tips to help you craft the perfect customer feel survey. In this post, we'll assistance you identify and gear up biased survey questions, and so that you tin avert inaccurate results due to poor question phrasing.

Here are seven common examples of biased survey questions, and how to ready them for your client experience survey.

1. Leading questions

Leading questions sway folks to answer a question i fashion or another, as opposed to leaving room for objectivity. If you lookout man legal dramas, you're probable already familiar with leading questions. Afterwards watching his witness get harangued, the lawyer bug an objection for "leading the witness," or putting words in the witness'south mouth.

In a courtroom setting, leading questions are usually filled with item and suggest what a witness has experienced, as opposed to letting the witness explain what happened. In the context of a customer survey, yous want to let your customers requite an authentic account of their experience, instead of dictating how they should view it.

Identifying a leading question

You can ordinarily identify leading questions by looking for subjective adjectives, or context-laden words that frame the question in a positive or negative light.

While a leading question may be a bit more innocuous in your survey situation than it is on the witness stand, it'due south withal important to avoid a leading question and so that you accept unbiased survey results. Here are some examples of leading questions, and how to fix them:

Leading question: How keen is our difficult-working customer service team?
Fixed: How would you describe your experience with the client service team?
Leading question: How crawly is the product?
Stock-still: How would you rate this production?
Leading question: What bug do you have with the design team?
Stock-still: How likely are you to recommend working with the design team?

Each of the examples of biased survey questions listed above contains a judgment, implying that the customer service team is "slap-up" and "hard-working," that the product is "awesome," or that you have "problems" with the blueprint squad. The corrected phrasing, on the other hand, is more objective, and contains no insinuations.

Leading questions are often unintentional, only if customers perceive your questions as manipulative, your simple leading question could atomic number 82 to a higher survey drop-off rate, a negative impression of your company, or a severely biased set of responses. Just one of these outcomes tin can significantly undermine the cease outcome you're working towards with your customer feel program.

Fixing a leading question

When writing your list of survey questions, phrase your questions considerately, and provide answer scales with equally counterbalanced negative and positive options.

ii. Loaded/Assumptive questions

The goal of your survey should be to get an honest response that will offer insight and feedback into the customer experience. A loaded question contains an assumption well-nigh a customer'due south habits or perceptions. In answering the question, people also inadvertently end upward agreeing or disagreeing with an implicit statement.

Consider this question: "Will that be greenbacks or credit?" The assumption in this loaded question is that the customer has already fabricated the decision to purchase. If they answer, they're implicitly like-minded that they will purchase.

Notwithstanding, if this question came after a client had already expressed a desire to buy, it wouldn't feel loaded at all. With loaded questions in customer satisfaction surveys, it'due south usually about context, and whether you lot've properly taken client data into consideration.

Identifying a loaded question

Checking for loaded questions can be tricky. Make sure you're reading through the entire survey, since context for the assumption you lot're making may come from a previous question, or from your customer database.

Here are some examples of loaded questions:

Loaded question: Where exercise you lot enjoy drinking beer?
Required qualifying information: That the customer drinks beer
Loaded question: How often practice you exercise twice a day?
Required qualifying information: That the customer exercises, and that they do twice a day

Considering these types of questions are often context-based, fixing these mistakes doesn't always require rephrasing the question. Instead, ensure that your previous survey question or existing client information qualifies the potentially "loaded" question.

For instance, you could inquire if your customer exercises twice a twenty-four hour period start, so ask how often they exercise and so. If they don't actually exercise twice a mean solar day, use conditional skip logic and so that the customer doesn't need to answer the irrelevant question.

Another manner to avoid a loaded question is past fine-tuning when you send your survey. For example, if you lot'd like to know why an ecommerce shopper prefers your spider web experience to a competitor's, yous wouldn't ask that question while they're browsing your site — that'd exist besides early. Y'all'd pop the question in a web survey on your checkout page, when you know for certain that they've chosen to go with you.

Also cheque the answer options as well equally question phrasing. Some loaded questions can be mitigated by providing an "Other" or "I do non ____" answer option every bit a way to opt out.

Fixing a loaded question

Don't make unfounded assumptions about your customers. Make sure you're qualified to enquire the question, skip the question if it's irrelevant, or provide an reply option that the customer can utilize to tell you lot that the scenario isn't applicable to them to avoid survey question bias.

three. Double-barreled questions

To understand the double-barreled question, just call back of a double-barreled shotgun. Information technology shoots from two barrels in one go.

In the realm of biased questions, that double-barreled barrage is really a convoluted question involving multiple bug. By asking ii questions in one, you make it difficult for customers to answer either one honestly.

Identifying a double-barreled question

When yous proofread your questions, check for "and" or "or." If you demand a conjunction, chances are you're asking about multiple things.

Double-barreled product question: Was the product easy to find and did you purchase it?
Fixed case (part i): The store made it easy for me to find the product.
Fixed case (part 2): Did y'all buy a product from our visitor during your final visit?
Double-barreled onboarding question: How would you lot rate the preparation and onboarding process?
Fixed onboarding example (part i): How would you charge per unit the training materials?
Fixed onboarding instance (part 2): How would you lot rate the onboarding process?

As y'all tin can run across in this biased question case, the fix for a double-barreled question is to separate it upward. Doing this has ii benefits: your customers don't get confused, and you can interpret the results more than accurately. After all, in the product case, if a customer had answered "yes," which question would they be answering? That they found the product hands, or that they purchased it? You lot would have no way of knowing via the survey results.

The other fix would be to inquire merely the question that meets your survey goals. Are you lot looking for feedback on how well your store is laid out? For the onboarding question, would yous like feedback on the overall onboarding process, or just 1 part of information technology — the training materials?

Fixing a double-barreled question

Ask ane question at a time. Don't overcomplicate. Check out this guide on how to avoid double-barreled questions in your surveys to collect accurate feedback.

iv. Jargon

Jargon is a give-and-take or phrase that is hard to understand or not widely used by the general population. For your customer survey questions, proceed your linguistic communication unproblematic and specific. Don't include slang, catchphrases, clichés, colloquialisms, or any other words that could exist misconstrued or offensive.

If y'all ever need to survey customers in multiple languages or locations, removing jargon will also make your questions easier to interpret and more readily understood.

Identifying jargon in your survey question

The worst office nearly jargon is that you may not even realize you lot're using information technology — those phrases could be embedded in your company culture. To avert confusing language, try to discover people from multiple historic period groups or demographics who are not familiar with your company to test your survey.

Product question with jargon: The product helped me meet my OKRs.
Fixed: The product helped me meet my goals.
Service question with jargon: How was confront-fourth dimension with your customer back up rep?
Fixed: How would you rate your experience with [team member]?

What are Objectives and Key Results (OKRs)? How do you know your customers use that system to assess their goals? Does face-time refer to the Apple video chat app, or does it mean you just spoke with a customer service team member in person? If a customer needs to spend extra time understanding the question, they may stop taking the survey altogether, or they won't be able to answer well.

Fixing a question with jargon

Proofread and exam your survey with an eye for removing all confusing language. Acronyms are a telltale sign that your survey may contain jargon.

5. Double negatives

While yous're checking your survey questions for jargon, don't forget about proper grammer. A double negative occurs when you employ ii negatives in the same judgement. For example, simplify "Don't not write clearly" to "Write clearly."

Avoiding a double negative may seem basic, but when you're in a rush and trying to get your survey out, it'southward easy to miss. It's as well quick to fix if you know what to expect for.

Identifying double negatives

Check for double negatives by looking for instances of "no" or "not" paired with the following types of words:

No/not with "un-" prefix words (also in-, not-, and mis-)
No/not with negative adverbs (scarcely, barely, or inappreciably)
No/not with exceptions (unless + except)

Here are some examples of double-negatives, and how you tin can edit them out.

Double-negative: Was the facility not unclean?
Stock-still: How would you charge per unit the cleanliness of the facility?
Double-negative: I don't scarcely buy items online.
Fixed: How often do you buy items online?
Double-negative: The website isn't like shooting fish in a barrel to use unless I utilize the search bar.
Stock-still: The website made it easy for me to notice what I was looking for.

Errors like double negatives are easier to grab if you read your survey questions and answers out loud. In one case you fix them, your survey will be easier to understand.

Fixing a double-negative

Two negatives brand a positive, in the sense that they cancel each other out. To correct a double negative, rephrase the question using the positive or neutral version of the phrase.

6. Poor answer calibration options

We've gone over quite a few examples of biased questions, only the question isn't the simply thing that tin be biased: your respond options can crusade bias, too. In fact, survey answer options are just as important as the questions themselves. If your scales are confusing or unbalanced, you lot'll terminate up with skewed survey results.

In general, carefully consider the best manner to ask a question, and then retrieve about the response types that will nearly effectively allow your audience to offer sincere feedback. Cheque that your answer options lucifer these criteria as yous craft your survey questionnaire.

Identifying mismatched scales and poor answer options

Sync the answer calibration back to the question
For example, if you're asking a question about quality, a binary "yes/no" answer probably won't be nuanced plenty. A rating calibration would be more effective.

If you ask "How satisfied" someone is, use the word "satisfied" in your answer scale. If you're using a smiley face up survey, inquire your customers how happy they are.

Mismatched respond scale example: How easy was it to login to the company website? Answer: Yes | No
Fixed answer scale: The login prompt made it easy for me to log in. Answer scale: 1 – Strongly disagree | 2 | 3 | 4 | 5 – Agree

Proofread for mutually exclusive reply options
If yous're using multiple choice answers, proofread the options to make sure they're logical and don't overlap. This issue often occurs with frequency and multiple choice questions. For case:

Frequency question: How frequently do y'all check your email in a day?
Overlapping answer options: A. 0-1 time | B. 1-2 times | C. 2-three times | D. More than 3 times
Multiple choice question: What device do yous commonly use to check your electronic mail?
Overlapping reply options: A. Computer | B. Mobile Telephone | C. Tablet | D. iPad

Which option does someone who checks their email once, twice, or three times a solar day choose? An iPad is a tablet, and then which would you select? Always double check your reply options to remove redundancy and brand certain the categories are mutually sectional.

Cover all the likely use cases
For multiple option questions, ensure your answer choices cover all the likely use cases, or provide an "Other" option. If your customers are constantly choosing "Other," it'due south a sign that y'all're non covering your bases.

Double check your survey answer option functionality
If your survey question says yous can "Check all the boxes that apply," make sure people actually can select multiple options.

Utilise balanced scales
If you lot're creating your own scale instead of using a templated agreement or satisfaction scale, make sure the lowest possible sentiment is at the bottom, that the best is at the acme, and that the options betwixt are equally spaced out.

Survey question: How was our service today?
Unbalanced scale: Okay | Adept | Fantastic | Unforgettable | Listen-blowing
Survey question: How satisfied were you with our service today?
Counterbalanced calibration: Very dissatisfied | Dissatisfied | Neutral | Satisfied | Very Satisfied

What'south the difference between "okay" and "good," or "unforgettable" and "heed-blowing?" Is "okay" really the worst possible experience someone could have? With a scale like this, information technology would be impossible to get useful insights.

Be consistent with scale formatting
If you lot're using rating scales throughout your survey, be consequent about which end of the scale is positive. For instance, if you're using one to 5 scales to rate agreement or satisfaction, make sure the positive end is always the highest number on the right.

Fixing your answer scales

Read through your questions and answers, and take your survey for a examination run. Make certain your answers direct correspond to the questions you're asking, and that your answer options don't overlap.

7. Disruptive answer scale formatting

In that location's a strong likelihood that customers will exist responding to your surveys from various devices, whether that'south on desktop, mobile phone, or tablet. Your reply calibration formatting needs to take this into account, and so the rating options are easy to scan and understand.

Checking for user-friendly formatting

Earlier you deploy your survey, check how it renders on desktop and mobile for various screen widths. Endeavour to make sure reply scales fit nicely inside the frame.

example of a poorly formatted rating scale that biases towards negative responses — Examples of poorly formatted rating scales

Since people tend to skim, yous desire to make sure your answer scales are formatted as intuitively as possible. When the rating scale wraps to the next line, customers may mistakenly tend towards choosing "5" because it's the right-most option, and haven't scanned to the second line, leading to depressed scores.

Examples of properly formatted rating scales

For mobile, endeavor to have the positive terminate of the scale at the top, since people are used to associating a higher score with a tiptop position.

Fixing answer scale formatting

When you design your survey, effort to make it responsive to screen width and so that the scales always render as you want them to. Note that not all customer feel survey platforms are created equal, as some are more than optimized for multiple devices than others.

Delighted surveys are optimized for every device, so customers can respond easily and accurately.

Summary of survey questionnaire best practices

Preventing biased survey questions from slipping into your survey is easy if yous know what to look for. Virtually of the time, it'due south virtually taking a few actress minutes to read through your questions to brand sure everything makes sense.

To review, hither's a quick checklist of dos and don'ts for a foolproof survey questionnaire:

Don't pb the witness: avert leading questions
Don't brand assumptions: avoid loaded questions
Don't overload your questions: avoid double-barreled questions
Don't use confusing language: avert jargon
Practice write conspicuously: avoid double-negatives
Practise check your answers: sync your answer scales to the question
Do consider the user experience: format your surveys for all devices

It's worth noting that survey bias can be acquired by more than only question and respond phrasing. Learn more in our complete guide to survey bias examples.

For a rundown of the entire customer experience survey process, from setting goals to reporting on results, check out this survey design guide.

Almost of the steps for crafting successful surveys involve conscientious planning and execution, but having a tool that supports the implementation of a successful customer experience survey strategy e'er helps.

Delighted offers a consummate solution, with proven client feel survey templates and customizable follow-up questions. Sign upwardly for a Delighted free trial and start getting client feedback using our experience direction software.

Lucia is a marketing professional with ten+ years of experience in creating content that educates, inspires, and converts. In her spare fourth dimension, you'll discover her trying out new recipes, propagating houseplants, and traveling the world. More