Ranking Scales vs. Rating Scales: Which Works For You?
• • • • • •
At Suzy, we offer multiple question types, methodologies, and quant and qual in one integrated and consolidated platform. With so many choices, deciding which questions and methodologies work best for your goals can be difficult. And the types of questions we ask can have a profound impact on the results – both positively and negatively.
In order to create products our consumers will see and love, it is more important than ever to produce surveys that generate significant and useful results.
Rating scales and ranking scales are important tools that, while seemingly similar, serve very different learning outcomes. As part of the Suzy ecosystem, rating and ranking scale questions each have a unique role in research. Let’s take a look at how rating and ranking scale questions work, how they differ, and where they belong in your survey to get the best feedback from your consumers.
What is a Rating Scale?
Rating scale questions are a variation of multiple choice. They ask the respondent (or members, as we refer to our consumer panel at Suzy™) to assign a value to a particular object or subject. Rating scales are close-ended questions that can help you gain quantitative data – information you can measure, hard facts. Rating scales allow you to collect data in a way that is easier to analyze and use.
A Rating scale question is one that seeks respondent feedback in a comparative form for specific features, products or service – “on a scale of 1 to 7 where one means ‘not at all likely’ and seven means ‘extremely likely,’ how likely are you to purchase the product in the next 3 months?"
Say, for example, you want to determine how your customers feel about your new soda branding. Using a rating scale question, you can show them a photo of the newly branded soda bottle and ask how likely they are to choose your item: very likely, neutral, or not at all likely.
Rating scales can take on many different forms. A numerical scale gives respondents a series of numbers to choose from, each representing a different rating.
Comparison scales, or simply scale questions, on the other hand, ask respondents to compare two or more items on the same scale. For example, if you want to determine whether or not quality or price makes more of difference when consumers purchase certain products, you could use a comparison scale to do so.
How influential are the following factors, when purchasing product X?
1. Quality of product
2. Price of the product
Scale: 1- Not at all influential, 7- Very influential.
If you wanted to determine how the cost of your soda compares to a competing brand, you could use a custom grid question. Then, ask respondents to determine if yours is priced higher, about the same, or lower.
The Likert scale is one of the most often used rating scales and one you probably recognize. In this method, members are asked to agree or disagree with a statement. Combined with a numerical list, it can often look like this:
On a scale of 1 to 7 (one being strongly disagree and seven being strongly agree), rate your opinion of each of the following statements.
What is a Ranking Scale?
Ranking scales offer a different approach to gathering data—these questions ask respondents to compare items to one another, rather than rating them on a common scale. When trying to negotiate which items to remove from your dessert menu, for example, you might ask customers to rank the seven desserts you offer from their most favorite to least favorite, giving you insight into customer preferences.
Let’s consider your hypothetical soda brand. Say you want to collect data on how people respond to several different branding options. You present respondents with five different images of your branding options and ask them to rank the images in order of most preferred to least preferred.
This style of question gives you insight into what your customers like or dislike and provides you with feedback of how your products (or branding, in this case) compares alongside others.
A rating scale questions allows you to measure strength of response. A ranking scale question allows you to measure priority of options. Using the two in tandem can give you very powerful insights into consumer preferences.
For example, if you were to ask a member to rate five different product concepts and she rated three of the five with a score of “7,” you do not know which of the three is most important. Following this question up with a ranking question of the five options would give you this critical information.
Ranking vs Rating: Which is better?
Ranking and rating scales each have their advantages. They also both have a significant role in a survey. Neither question style can produce the best results on its own.
The most accurate surveys combine both styles of questions, along with open-ended questions. But getting the most out of your survey isn’t just about knowing which type of question to use. It also requires knowing when each style of question is appropriate.
Rating Pros and Cons
Rating scales are one of the most commonly used survey questions, and for good reason; because rating questions are a variant of multiple-choice, they are often clearer and simpler for respondents to understand.
Rating scales are best to use when you want to measure the performance of something or someone. Questions can vary widely.
How likely are you to recommend our products to a friend or colleague?
Rate your level of satisfaction with our customer service.
To what degree did our onboarding process positively impact your success as an employee?
Rating questions are great when you are trying to determine how well a product is performing, how consumers feel about a new product, or where they perceive there might be a gap in the market.
One of the advantages of rating scales in R&D is that it allows your respondents to not only offer their preference, but also assign the same value to multiple items.
However, this can also be a disadvantage. Some respondents won’t ever choose the highest (or lowest) rating. Others will automatically assign every question the same value.
It’s also important to keep certain biases in mind when it comes to rating as well, including:
Scale Interpretation Variability: Respondents may interpret scale points differently, leading to inconsistent data.
Central Tendency Bias: People often avoid extreme ends of the scale, skewing responses toward the middle.
Acquiescence Bias: Some participants may agree with statements regardless of their content, biasing your results positively.
Ranking Pros and Cons
The weakness of the ranking scale is also its strength: It forces consumers to place more importance on one item over another. There may be multiple items, however, that consumers value equally. Ranking does not disclose that information.
If you are trying to determine how certain products compare alongside one another, ranking is the perfect way to pose your question. Instead of only gauging the level of satisfaction a consumer has in a single item, ranking allows you to see the unique value of several different items together.
Remember our soda branding example? Many famous soda brands have spent millions over the years continually rebranding, to the detriment of sales and public opinion.
These mistakes can be avoided. Using surveys to poll a consumer-based audience dramatically impacts the rollout—or potential rollout—of a new product. Say your soda company, for example, would like to introduce several new flavors. Once you have completed a rating survey to determine which flavors are most preferred, a ranking survey will allow you to discover the best order in which to release your new flavors.
"Rank these three flavors in order of most favorite to least favorite." Since you already know the flavors are well-liked, the rank of favorites will give you an idea of which flavor to release first, or which one will likely do the best.
In some cases, ranking or comparison questions reduce response and language bias in cross-cultural surveys. This can be especially useful when determining how a product might be received by a diverse range of consumers across the country and in different regions.
Ranking scales come with their own set of issues, however. Pose too many categories and respondents get mentally taxed. Questions go unanswered or ranking is not clearly thought through.
A Combined Effort
Using a combination of questions in your survey really does bring about the best results. Ranking is often a great way to narrow down data that can then be turned into a rating scale question.
For example, you might ask your consumers to rank the 20 items available in a vending machine from least to most favorite. Based on the results, you can then ask customers to rate how likely they are to purchase each of the favored items.
By using questions together, you gain a more comprehensive understanding of what your consumers like, what they want, and how they make decisions.
How to Get the Most Out of Your Survey
When it comes to research and development, top companies indicate that consumer feedback is one of the most significant contributing factors to getting the most out of their R&D budget and their products.
How do you create a survey that will garner quick and useful insight?
The answer is in the question. By combining a variety of question types, and retargeting questions that require more in-depth data, you can get incredible on-demand consumer feedback in minutes.
Customize your questions
With Suzy™, it is easy to create both rating and ranking questions using the multiple-choice option. You can also add open-ended questions to your survey, giving you further insight into your respondents or members.
When you’re ready to rebrand your packaging, use a rating scale to ask members to give feedback on your proposed packaging. Or, better yet, give them several options and ask them to pick their top three or four.
Suzy™’s platform provides room to customize questions to get a better response from your members. You can randomize answer choices to ensure members aren’t influenced by the order, or allow users to choose more than one option in order to see a partial ranking of several items.
Retarget your questions
Sometimes finding the right question takes trial and error. With Suzy™, you can retarget your survey audience to ask more specific questions based on previous responses.
Creating a great survey or a great question starts by narrowing your results. Ask your consumers an open-ended question or a ranking question about your product. Use those results to narrow down your questions.
In the same way that ranking questions can inform how to word a rating question, your previous surveys on Suzy™ can inform how to retarget and narrow your results.
Retargeting also opens up options for honing in on your target audience. Rather than rebranding your soda can, let’s say you are ready to launch that new flavor of soda.
Asking your members which type of soda they drink (classic flavors vs more niche flavors) can help you fine-tune your audience. Next, retarget those who like the unique flavors and ask them more specific questions related to your new product.
Revolutionize Your R&D with Suzy™
As competition increases, companies are looking toward R&D more and more to stay on the cutting edge of innovation. Real-time consumer feedback is the key to staying on top of things. Suzy provides responses to survey questions in just minutes, meaning you can get feedback even in the midst of ideation and development.
Suzy allows you to bring your best rating scale and ranking scale questions to the table and present them to a wide-array of members in a simple, easy-to-use format.
Instead of wondering how to reach your consumers, ask yourself what types of questions will generate the best feedback from your audience. The results will speak for themselves.
Want to see how they work? Book a demo with our team today.
