Just this week a new report showed that mobile now accounts for over one third of all internet traffic at a global level. It’s honestly kind of old news. Certainly not worthy of a blogpost; if you’re in market research you had better know how quickly mobile is overtaking desktop in internet access and usage statistics.
But researchers at a global level sure aren’t transitioning as quickly as savvy readers would think. How a brand conducts its research can be an important part of the overall customer experience. If people are viewing most information about a brand on their mobile devices, why are the surveys about these brands not following suit?
Many surveys conducted in the industry are still only answerable on desktop computers, though mobile penetration is certainly continuing to grow via automated device rendering of online surveys for phones and tablets. It’s an upwards trend for sure.
But what about sample representation compromises that are made by taking this step? This is likely one of the primary concerns of the researchers clinging to single-channel surveys. If respondents can complete a survey on the device of their choosing, does affect our ability to collect a representative sample or the quality of data?
The team at Research Now recently carried out a study of 8,000 people across the UK, France, Germany, Spain and Italy to try to find out. The research tested whether there were significant differences in the responses to a questionnaire, depending upon the device used. There are very obvious differences that could come into play: screen size and orientation, selection interface, typing interface, and the quality of internet/ data connection. What they wanted to examine was whether the differences between devices have a direct impact on how participants interact with them, how they interact with a survey and then, what conclusions can be made.
Among other things, the study examined the impact of sliding scale length, question length, and the number of open ended questions. The key objective was to understand the impact the device used to answer a survey has on the data collected, and to establish guidelines to minimize data inconsistencies.
The results of this study suggest that device screen size does not seem to have an impact on responses to different scale lengths. However, the way in which the scale is presented does. We looked at scale questions presented with radio buttons, sliders or grid bars. Answers provided using a slider exhibited a greater number of significant differences across devices, whist grid bars and radio buttons achieved greater consistency across device.
As might be expected, participants answering an open ended question via desktop will type in longer answers than those on mobile devices, but the difference isn’t as significant as one might think. However, because drop-out rates for those taking part on a mobile are twice those on a desktop, it’s recommended that researchers keep open-ended questions to a maximum of three. Answering these questions is time consuming, and survey length should be kept to a minimum on mobile.
Overall, the study found that the quality of research data is not adversely affected by any one device. So brands can get robust, actionable insights from consumers via the device that suits them best.
Conducting research across multiple devices makes it easier to include all demographics. Excluding mobile (or desktop) from your research means exclusion of a defined population segment. By choosing multichannel, you’ll have a better chance to understand audiences, produce better insights, and make better business decisions.