How to Make a Conjoint Analysis Survey1

Survey ranking questions

SHARE THE ARTICLE ON

Table of Contents

What are ranking questions?

Pretty obvious from the name itself, ranking questions in a survey lets the respondents rank the list of options based on their views and opinions. 

Survey ranking questions provide quantitative data and allows the respondents to present which presented options they would prefer the most and which the least. This questions provides a close-ended scale and the respondents compare the options. 

A ranking scale is nothing but a close-ended scale that uses a row and column approach for the items to be compared against each other. Survey ranking questions use ranking scales for questions regarding product features, preferences, expectations and more.

Transform your insight generation process

Create an actionable feedback collection process.

online survey

Types of survey ranking questions

In this section we will see what the types of survey ranking questions are:

  • Drag and drop ranking

Respondents are given a list of options and they have to just drag and drop the options one by one in another provided space. The options are ranked one after the other where the first means highly preference and the last means least preferred. 

Example: Rank the performance of our website according to your views

  • Radio button ranking 

Radio button ranking looks somewhat like a metric with each option having its own ranking scale. 

Example: Rank our product service according to your experience

  • Text box ranking 

In this type of survey ranking question, every option has a textbox against it where the respondents will type their preferences for the particular option.

Example: Rank your most favourite to least favourite series

  • Select box ranking 

Similar to the drag and drop ranking, the select box allows the respondents to shift the options up and down so they are in an order that represents their preference. 

Example: Rank the restaurants according to your preferences

Download Market Research Toolkit

Get market research trends guide, Online Surveys guide, Agile Market Research Guide & 5 Market research Template

Making the most of your B2B market research in 2021 PDF 3 s 1.png

Analysing the results

Ranking questions are analysed by calculating the average rank of each item. This determines which item was most preferred and which was not. 

The ranking average is calculated as:

Where 

w = weight of the ranked position

x = response count for answer choice

The application of weights works in reverse where the highly preferred item (#1) has the highest weight and the least preferred item has the lowest weight. Weight count depends on the total number of items in your question.

Example: You have 5 items in your ranking question, the weight of each of them based on their preferences will be:

  • The #1 choice weights 5
  • The #2 choice weights 4
  • The #3 choice weights 3
  • The #4 choice weights 2
  • The #5 choice weights 1

Survey ranking questions tips

  • It is a good practice to limit your ranking scale to 6-10 items per question. 
  • It is ok to sometimes ignore the middle rankings. Like on a scale of 10 items, the 4th and 5th rankings aren’t significantly strong or reliable. 
  • Use broad categories that will cover all the small items or features you wish to present in the question. This will reduce the item count.
  • Make sure you experiment enough with different types of ranking questions to find which one best suits your topic.
  • Run your ranking questions through mocks and trials to make sure they provide suitable results.
  • Avoid forcing the options to your respondents especially when you know not all of them have all the information related to the topic. 
  • Don’t consider ranking questions if you are looking for value-added results. Rankings only tell how what people prefer but not why they do. In such a case, it is better to use qualitative questions.
  • It is advised to follow up ranking questions with a rating question just to check the strength of preference. 

See Voxco survey software in action with a Free demo.

Advantages of using ranking questions in your survey

  • Advanced customer insights – ranking provides you with not only what the customers like and dislike but also how much they value each of them. 
  • No response bias – unlike rating items, ranking them will serve a more sorted out preferences and avoid respondents just blindly answering the questions.
  • Ease – ranking questions are easy to instruct and use and save a lot of time with different types. 

 

Disadvantages of using ranking questions in your survey

  • Analysing ranking question surveys can be pretty challenging as the process is not easy.
  • In some way, it makes it look like the respondents are giving importance to one item over the others just because it is on the top.

Explore all the survey question types
possible on Voxco

Read more