Surveys aren’t dead: complementing Big Data with human feedback

Less than five years ago, the talk began about the full integration of passive digital data collection (mostly framed as ‘Big Data’) into business insights programs. And as often happens with new trends, a many insights groups dove headfirst into Big Data, abandoning the traditional survey in favor of the more automated digital behavior tracking. In doing so, they lost out on a ton of valuable insights that could only be gathered from asking questions to real people. Their data was suddenly missing the ‘why’ – the motivation behind the actions they were tracking.

Now in 2017, many of those insights groups have returned to surveys and settled on a complimentary solution that pairs passive digital behavior tracking and feedback surveys. Surveys are anything but dead, as they still offer incomparable insights into human behavior. Insights that cannot be matched by the automated collection of passive data. We have watched as the same organizations who shied away from surveys just a few years ago have started coming back and seeing the huge value in pairing their existing passive digital data collection with surveys.

One recent example that triggered this blogpost was when Voxco paired with C2 Montreal to complement their existing automated data collection with surveys that gauged event attendee satisfaction levels:

As one of North America’s most innovative business events, C2 Montreal deployed a significant passive data collection strategy. Smart technology embedded in participant badges tracked attendee participation levels, their purchases, new business networking connections, and more.

But the technology couldn’t track the emotions behind the actions. Why did people do what they did?  And how did they feel while doing it? That’s where a survey solution became essential. Voxco survey software helped the C2 team gather 1,200 completed post-event online surveys and another 1000 in-the-moment, face-to-face surveys. The attendee feedback served as the perfect complement to the automated data collected from the smart badges.

The two data types worked together to paint a live picture at numerous moments during the event. Event organizers could see a live snapshot of what participants were doing, and how they were feeling. Then post-event, organizers could look back at a robust portrait of all attendee activities and pair that with a complementary overview of how participants felt about each individual event element.

So to this pessimistic industry in which we all work, how can we emphasize the importance of the traditional survey versus the passive data collection of ‘Big Data’? By emphasizing that the choice is not about one over the other: it’s about choosing both.

Get in touch with a Voxco rep today to discuss how the industry’s most flexible survey technology can compliment your existing passive data collection processes.

6 ways technology can help you tackle sensitive survey questions

Members of the market research industry find themselves fighting tooth and nail for decent response rates and representative sample. So when your study is about drug use, sexual behaviors, political and religious beliefs, or other sensitive topics, you’ll need to ensure that you’re doing everything possible to keep response rates steady.

Asking sensitive questions via surveys or interviews can be considered intrusive or offensive, regardless of what the respondent’s answer may be. And if the respondent feels that their response could be contrary to popular societal norms, they may be more inclined to provide a dishonest answer that conforms to their perception of normal (this is called ‘social desirability bias’). Compounding the issues of question intrusiveness and social desirability bias, respondents may not trust that their responses will remain anonymous, or that the data will be kept secure.

Sensitive questions can negatively affect three important survey measurements: overall response rates, single question decline rates, and response accuracy – due to a higher percentage of respondents who answer sensitive questions dishonestly. That means that the inclusion of sensitive questions in a survey needs to be handled in an intelligent and delicate way whenever possible. Fortunately, technology can help.

The chosen survey channel can compound the issues around sensitive questions. The selected channel can exponentially increase respondents’ hesitation to answering sensitive questions:

  1. Self-completion surveys (e.g. Voxco Online): in which respondents complete surveys on their own time and in privacy;
  2. Telephone interviews (e.g. Voxco CATI): in which survey call center interviewers ask respondents questions over the phone;
  3. Personal interviews (e.g. Voxco Mobile Offline): in which interviewers use mobile devices to ask respondents questions directly in a face-to-face setting.

In general, respondents are more likely to answer sensitive questions honestly via self-completion surveys. But there’s tremendous value in choosing personal interviews over self-completion surveys, or choosing a complimentary multichannel approach, so interviews should not be ruled out immediately.

Here are six ways to use technology to boost response rates and response accuracy on sensitive questions:

1. Channel selection

As we mentioned above, self-completion surveys that can be taken by respondents in privacy offer a better atmosphere of anonymity which helps respondents feel more comfortable answering sensitive questions. The elimination of an interviewer can also reduce the social desirability bias introduced by the presence of another person.

2. Multichannel studies

Your studies don’t need to stick to only one survey method. Using an integrated survey platform, you can target a single respondent database and follow sample across multiple channels. For example, after a phone interview is completed, invite a segment of the respondents to complete an online survey. The logic of the online survey can trigger follow-up questions based on their CATI responses and probe into more sensitive topics that respondents would be less likely to confide to an interviewer.

3. Live channel switching

Consider redirecting interviewees mid-interview to a self-completion channel to ask sensitive questions. This temporarily gives respondents the anonymity and privacy needed for sensitive topics. Redirect CATI respondents to an IVR system to answer a few sensitive questions using their phone keypad, then return them to the interviewer to finish the survey. Pass the CAPI tablet to respondents directly and let them enter their answers directly into the questionnaire without having to say the words aloud to an interviewer.

4. Live question wording changes

As your survey progresses, be sure to keep an eye on real-time response analytics. If you note significant drop off rates at a specific question, or higher-than-average item nonresponse rates on specific questions, take action! Alter question wording, and instantly push the updates live. Add in some language that emphasizes survey anonymity and data security when surveys start veering into sensitive territory. That’s the benefit of live survey updating.

5. Survey logic/flow changes

If early results show that respondents are being turned off by sensitive questions, you could also adjust the logic and flow of the survey. Move sensitive questions further into the survey to build more trust and rapport with respondents before they get to them. If you have a sensitive question earlier in the survey, trigger ease-in questions and language to show only for those respondents who chose ‘refuse’ on the first sensitive question. Or go the other way: build trust with those respondents who did answer the sensitive question by hiding demographic or identifying questions at the end of the survey that could make respondents feel that their responses will be connected back to them.

6. Data hosting

Sometimes it’s enough for respondents to know that the survey is anonymous. But other respondents could want assurances that their data is being stored with the highest possible security and encryption, or on internal servers versus stored “out there” in the cloud. So give them those assurances.

Use technology to maintain response rates!

In a period of declining response rates, it is likely that respondents will be even more reluctant to take part in surveys that tackle sensitive topics. And in an age of growing concerns over data security, those who do respond to your surveys could be less inclined to reveal potentially embarrassing information about themselves.

Even in self-administered surveys, respondents may still misreport or refuse answers on sensitive questions. But there are a number of ways to use survey technology to your advantage when asking the tough questions. Get in touch with the team at Voxco if you’d like to discuss more unique ways to use a flexible survey platform to collect sensitive respondent data.

Complementing Online Surveys with Offline Interviews

Face-to-face interviews are a very different beast from self-completion surveys. We’ve gone into detail before about the best uses for each here and here. Unsurprisingly for an industry-leading provider of multichannel survey software, we concluded each post with a solution of ‘combining both is best’. And it really is. But how?

For market research firms running major studies seeking general opinions from a larger population, it can make a lot of sense to use offline interviewing as a complimentary secondary channel to the primary online channel. What do we mean by that? Well online is often the cheaper option, and a far easier way to get a survey out to a lot of respondents quickly. But to vary your sample and expand on your insights, adding a face-to-face interviewing component can make a big difference.

Here are a few reasons to complement online survey projects with offline interviews, and the methodology required:

Sample variance

There are some people who won’t complete online surveys when they receive them in their inbox. They don’t have the interest, or the time, or their spam filter keeps them from seeing the invitations in the first place. There is also still a portion of the population who don’t have regular online access.

A portion of people who are unreachable online can be reached in the real world via face-to-face interviews. Complementing your online surveys with face-to-face interviews expands your sample size, and can improve sample representativeness.

Quantitative data depth

You’ve gotten responses from your online survey, and the results indicate that shoppers most often buy a specific product on weekends at the mall. Your results have now given you a timeframe and location to conduct face-to-face interviews, aiming to better understand an average shopper’s mindset at a key purchase time.

Asking further questions in a face-to-face, interviewer-led setting adds a whole new dimension of data which complements the existing online survey results.

On-site, in-the-moment feedback

Take a look at our case study for C2 business events. Online surveys formed the core of the feedback collection process. But on-location mobile interviews helped gather a more raw and in-the-moment emotional response, when hindsight and fading memory have little to no effect on their responses.

Get more well-rounded insights by balancing the standard post-event feedback with some in-the-moment emotional feedback.

Sample follow-ups

Depending on the study, and the rights that your sample/panel has offered, it’s possible to follow up with specific sample segments via face-to-face interviews after the online study is completed. The online survey can ask questions that help segment the respondents into relevant sub-groups based on demographics or behaviors. Following up with a specific sub-group based on their online responses helps complement the overall study results.

If you have a solid panel of respondents who have granted permission to follow up in-person, interviewing the same respondents who have completed the online survey can give you a richer set of insights from individual sample.

Offline Face-to-Face Interviewing

In much of North America, market researchers aren’t using face-to-face interviewing enough. Could this be because they’re seeing it as an either/or when comparing it to online surveys? Because it’s so much more when used as an ‘and’.

For the short-term, it seems clear that market researchers will continue focusing most of their studies on online surveys. But those who can find a way to complement those results with face-to-face interviews will greatly enhance the insights. Embrace the enormous potential of a truly multimode study.

5 Unique Uses for CAPI Personal Interviewing Survey Software

Adding an interviewer to the mix for survey research is tremendously useful. We’ve discussed that before.

But thinking outside the box, what can personal interviewing tools help you achieve? The benefits of a flexible CAPI tool go far beyond just conducting face-to-face surveys. The power of an advanced mobile data collection software opens doors to dozens of potential uses for researchers. For example:

1. A gateway to richer insights

Excuse me sir, would you like to try our new granola bar? That’s a question that stops a lot more people in the street than asking somebody to complete a survey. Begin with a taste, and work your way up to more. CAPI can act as a gateway survey that leads to more data down the line. A taste of a new product leads to a quick on-the-street survey. Those responses lead to an invitation to self-complete a longer online survey, or even to join a panel of loyal users. When it’s part of a richer multichannel survey system, personal interviewing software can be used as a first step on a longer path to getting more insights from more people.

2. Secret shopping

Your ‘respondents’ could actually be inanimate objects. And your ‘interviewer’ could be more of an observer. CAPI tools offer the perfect solution for managing a team of loosely trained secret shoppers. Assign the ‘interviewers’ specific stores and they can use a mobile device to answer questions about their observations (e.g. quantity of products displayed, shelf placement, service ratings, etc.). They can upload photos, videos and audio files to accompany their findings. Managers can deploy shoppers, update survey questions, monitor metadata, and view live results from a central location.

3. Self-completion offline surveys

It’s common to think of CAPI tools as purely interviewer-based. But collecting data via mobile devices in offline mode can also be used for self-completion surveys. At events or in stores, tablets can be affixed in high-traffic areas to encourage patrons to leave feedback. Tablets can be handed off to shoppers or passersby to answer a quick survey with no need for a WiFi connection. If your tool offers offline data collection and intuitive survey design, let respondents use it themselves and synchronize at the end of the day to analyze the insights.

4. Event dashboards

At events, CAPI tools can be used as an attention-grabbing results dashboard. Some of our clients have incorporated one or two fun questions into a longer face-to-face interview (e.g. ‘Who will win the Superbowl this year?’). As interviewers stop attendees to answer surveys on tablets, responses to those fun questions are regularly synchronized via WiFi with a results dashboard on an in-booth display. The results draw attention, start conversations between interviewers and attendees, and encourage participation in the larger survey.

5. Guided multilingual self-completion surveys

Collecting feedback at tourist hotspots can be difficult. There is a natural barrier between multilingual tourists and monolingual interviewers. But a good CAPI tool allows interviewers to seamlessly change the language of the survey, and turn the tablet around for respondents to answer questions directly. Just ask Cimigo Hong Kong, who have been doing it for years!

Who do you need to interview?

Think of the personal interviewing software as a set of outreach tools that help you meet more potential respondents, and get more varied data in the field. The best researchers don’t restrict themselves to standard methodologies. Voxco Mobile Offline offers engaging survey design, powerful interviewer management, and offline data collection. That’s huge potential for researchers who want to get unique results.

Are personal interviews better than self-completion surveys?

We recently outlined the benefits of letting respondents complete surveys with no guidance from interviewers. Self-completion surveys are the most common method for quantitative surveys in the market research industry, primarily due to the cost and ease of deployment.

But what about interviewer-led quantitative surveys? What are the benefits to having a professional guiding respondents through a survey (face-to-face or via telephone) and recording their answers?

Here are a few reasons to choose interviewer-based surveys versus self-completion:

Location-specific, moment-specific insights

In-person surveys are brought directly to the respondent, wherever they are. This is a huge advantage for interviewing consumers when they are still present in the specific place your survey references (e.g. mall, in home, tourist destination). Interviewers can even incorporate a full product experience, including touching, tasting, or viewing products in their natural setting.

Asking for feedback while respondents are still experiencing something will generally lead to richer responses than relying on their memories.

Interviewer guidance

In-person or telephone interviewers can provide assistance and clarity on question meaning. They can ensure the question was answered adequately. They can aid recall by prompting. And they can keep respondents motivated to remain focused on the survey.

Well-trained interviewers (and well-structured/well-written surveys) can accomplish all of this without biasing the survey results themselves.

Reach new respondents

In-person and telephone surveys allow you to reach groups with lower internet penetration. And friendly interviewers can motivate participation from some types of people who could be less likely to respond to online surveys/join online panels.

When beginning a project, survey project managers are faced with the task of reach. If their existing online survey panel/database isn’t specific enough to the required demographic, intercept interviews or regional dialing can reach a whole new world of respondents with no prior need for their contact information.

Lower respondent initiative needed

Once a respondent is reached in-person or on the phone, all they need to do is verbally answer questions as they’re posed by the interviewer. There’s no need for them to read the questions, or to manually manipulate the survey itself to provide responses.

Why not both?

The market research world is not black-or-white, and researchers fortunately don’t need to choose between exclusively online or exclusively in-person surveys.

Some studies need the speed and freedom provided by online self-completion surveys. Some need the richer level of respondent data that generally comes from interviews. Most researchers see the value of both; these methodologies can be packaged to offer the flexibility of channel choice on individual studies, and the power of a centralized multichannel survey database.

What’s right for you? Let us know and we’ll show you how the world’s most flexible survey software can fit into your methodology, and fits comfortably into your budget.

Voxco Online v5.6 – Restructured Navigation

We’re excited to be managing a full roll-out across all of our SaaS servers that gives users access to the new Voxco Online (Acuity4) platform version 5.6.

The big news is that this version includes the release of Voxco Mobile Offline, our personal interviewing survey tool. It is integrated directly into the same platform as Voxco Online to maximize multichannel efficiency. If you’re interested in a demo, get in touch with us!

Even for online-only clients, there are a ton of new benefits to v5.6. This update integrates many client-requested updates to the overall navigation and layout of the platform. Here are a few of the biggest changes:

Maximized screen space

Elements within the platform and questionnaire editor have been aligned to the edges of your browser window. We have also eliminated tertiary navigation bars so that only a primary and secondary navigation remain:

  • The primary navigation is a permanent link to the four most common sections of the platform
  • The secondary navigation serves as a breadcrumb trail, offering easy access and quick actions for any step along the path to your current page.

The result of these changes is a more optimized experience. More of your important data fills more of the window while you’re editing questionnaires, sample lists, reports, and more. So get to work! 😉

Reporting via top navigation

We added a general ‘Reporting’ tab to the permanent navigation bar. It also provides one-click access to reporting subsections via a drop-down. Reporting for all of your surveys is now quickly accessible from anywhere in the platform.

Panels via top navigation

Panel Manager can now be accessed via the top navigation for anytime access. Using the secondary navigation breadcrumbs, you can easily jump directly to individual panels, panelists, and quick actions (imports, invitations, templates, etc).

Merged users & user groups

We have merged the two user management sections (users, user groups) into one single page instead of two. You can now view at a glance which users are in which user group. On the left you can access all of your user groups. In the central portion of the page, view and access individual users from the currently selected user group. New users and user groups can be added via a single, prominent button in the top right corner.

Search boxes

All lists now have search boxes to simplify access to specific surveys, users, user groups, panels, reports, templates, and more. See the examples above for quick searching user groups or individual users.

New features

v5.6 also includes a handful of brand new features, many of which are client-requested, and all of which will improve user productivity. You can read about them in our blogpost on January feature updates.

Want more?

All of our SaaS clients are being upgraded to v5.6 in the very new future. Release notes are available that detail all of the above changes in more detail. On-premise clients can request access to 5.6 at any time. Anybody else who is interested in learning more about v5.6 or seeing the new Voxco Mobile Offline in action can get in touch with us today! 🙂

January Feature Updates

It’s still early in the year, but we are already making major updates to the industry’s most flexible survey software.

The beta release of Voxco Mobile Offline and its integration into the existing Voxco Online platform has triggered a round of updates packaged together as Voxco Online 5.6. This new version will further improve the intuitiveness of the navigation layout, and add a few new navigation elements that will improve user productivity. We’re communicating these changes directly with clients for now. We’ll update the blog on Voxco Online 5.6 and Voxco Mobile Offline availability very soon.

Along with the navigation improvements, we are also packaging a handful of additional features based on feedback from clients and our own internal team of survey experts. Here’s a list of the new minor features hitting with the release of Voxco Online 5.6:

Detailed notifications & execution history on tasks/distributions

When importing samples or exporting results, users can now receive a comprehensive summary of which tasks and distributions were executed. Select the ‘send notification’ option for tasks that you want to have tracked in these reports.

  • Distribution summaries: Task name, survey name, description, content, schedule, and numbers of sends and failures.
  • Export summaries: Task and file name, file size, survey name, description details, schedule, filter details, and numbers of successful imports or failures.
Adding RGBA transparency for color picker

The color pickers within the questionnaire editor and Look & Feel editor now support RGBA transparency, so that survey creators can set transparencies for selected colors within their survey designs.

Show only pending/running distributions

We have added quick filters so that you can easily see which distributions are currently being executed and which ones are still pending. This will be a benefit for surveys that have a lot of distributions.

Updated survey engine default doctype to HTML

The default doctype for the survey engine has been changed from XHTML to the more standard HTML. This doctype offers better compliance with accessibility guidelines (e.g. WCAG).

Results encryption using PGP on export tasks

When you’re exporting survey results, you can now apply PGP (Pretty Good Privacy) encryption. Users who employ PGP encryption software can now use this method when sending results via email or uploading to an FTP. Select the ‘Encrypt files with PGP’ option (highlighted below).

Remove curly brackets and spaces from response exports

For all formats (except Open-End), survey responses are now exportable without spaces and braces – { } – for all system variables. Select the ‘Remove Curly Brackets and Spaces Of System Variables’ option:

If you have any questions about any of these, or how they work, let us know! We are rolling these updates out along with Voxco Online 5.6, and communicating details to clients directly. Plenty more information still to come on Voxco Online 5.6 – stay tuned!

Online surveys are not enough

Voxco-9020

Part 2: Get surveys noticed in the offline world

You don’t have to read Part 1 of this blog series to be aware of the worrying issue facing market researchers: online surveys are being ignored in growing numbers thanks to survey fatigue in respondents. While online remains an essential part of any survey program, it’s becoming increasingly difficult to ensure that they are getting noticed – they’re just too easy for respondents to ignore.

So how can your survey stand out? By changing channels completely, and using the personal touch of an interviewer. There’s something to be said about the old-school survey method: it’s harder to say no to a real person’s interview request than it is to ignore one of hundreds of web links seen online each day. There is a reason face-to-face interviewing is often referred to as ‘intercept.’ You are literally intercepting a respondent in the middle of their day and getting noticed.

Adapt. Existing survey methods are in decline, and only the flexible will survive. Managing effective survey programs in 2016 requires you to complement online surveys with offline channels that can help create a more representative tapestry of insights. Here are some of the major advantages of complementing online surveys with other, interviewer-lead channels:

In-the-moment feedback

Most online survey invitations are sent after an event or experience. Imagine a customer satisfaction survey, automatically sent via email after a purchase: the respondent opens the email a few hours or days later and completes the survey, remembering back on their experience. But when a respondent is asked about an experience while they are still experiencing it, their answers are typically more emotional, and more direct.

Thoughtfulness of responses

When an interviewer is asking questions personally, they are better able to nurture the conversation and ensure that the respondents fully understand questions and carefully consider responses. Interviewers can ask follow-ups and add prompts to help elicit more robust answers. There is no way of knowing if respondents online are taking the time to consider their answers.

Respondent access

To survey respondents online, researchers either need their email, or they need to get their survey link into a respondent’s hands somehow (like via the bottom of a receipt). But how can researchers survey truly random respondents who haven’t yet engaged with a brand in some way? Imagine 10,000 people passing through a train station in the morning, or average people in a given city on a given day – how can a researcher get their survey in front of them without ever having interacted with them before? When your surveys are offline, respondents are suddenly everywhere.

The personal touch

In a world of online gadgets, a little offline communication can warm the soul. Offline studies see friendly interviewers approach respondents with tablets and smiles, looking for a genuine conversation. Will many people still turn down the request? Of course! But unlike approaching respondents by sending another email and another web link to click, is it so crazy to think that offline surveys could actually be a positive interruption for some people?

Believe the hype – the rapid evolution of survey technology offers incredible online survey opportunities. But today’s online-overdosed reality may actually require researchers to complement their online surveys with a step away from the web and back into the real world. It’s a fairly fundamental business concept: what’s different gets noticed. So think different.

Want to explore how easy it can be to create engaging offline surveys for real-world interviewing? Get in touch with Voxco today!

 

 

Survey Software Training: A Handy Checklist

Survey Software TrainingDiane, Voxco’s survey training specialist, is a road warrior with thousands of training hours under her belt. She has given Voxco training sessions all over the world, including Chicago, Paris, Milan, Sydney, Singapore….and many more places. Here is her checklist to help you get the most out of your CATI or CAPI software training process. Let’s dive right in….

BEFORE:

  • Pre-select your training questionnaires. The best survey projects to learn from are your own. A couple weeks before training, agree with your trainer which of your CATI/ CAPI questionnaires should be used in the training session. Choose something typical, but also be sure to cover more complex scenarios.
  • Book enough training hours. Discuss your needs with your trainer beforehand to block out a realistic schedule. It’s tempting to try to keep it as short as possible. But remember that long-term, it’s always better to have invested a bit of extra time in up-front training.
  • Check your equipment. We know this one sounds obvious but it happens—a room full of trainees, eager to learn, only to just then realize that the projector is out of order. Test your network connections, make sure trainees have their own work stations and yes, triple check that projector.

DURING:

  • Elect your software champion. Find a go-to colleague, preferably someone software savvy and invested in the project, to be the product champion. This person can help internal teams once the training is over.
  • Group up trainees. If you’re training a large number of people, prioritize different sessions to different groups based on their software knowledge. In our experience, optimal training sessions are achieved when group sizes are no more than eight participants.
  • Encourage questions. Staying on course sometimes means slowing down for a moment. Don’t be afraid to take some time to expand upon a certain topic. We really can’t stress this enough. When someone feels out of the loop, their first reaction is often to disengage. Encouraging questions keeps everyone interested and on the ball.
  • Embrace change. Training offers you the opportunity to revisit old techniques and explore things in a new light. Don’t let yourself get in the way of major productivity gains!
  • Be committed. This means no cellphones, emails or texting during sessions. Of course, work commitments don’t just disappear—which brings us to the next point…
  • Breaks. Breaks allow trainees to stay focused during training sessions and to catch up on other projects between them. Breaks also offer a good opportunity for trainees to get some one-on-one time with their trainer and address some things that may be unclear.

AFTER:

  • Practice. Practice. Practice. Now that trainees acquired new skills, don’t shelve them! It’s important that everyone put the software to use as soon as possible.
  • Knowledge transfer. Avoid knowledge gaps by having your software champion train new resources as often as possible.
  • Set refresher sessions. In our experience, the average trainee absorbs around 30% of their product knowledge during the initial training sessions. A lot more is then accumulated through doing (again: practice, practice, practice!). But one of the best ways to make sure everyone is working at full productivity is through refresher courses. A periodic two- to three-hour session with your trainer transforms software operators to software experts.
  • Best practice guide. Training teaches your team to efficiently use the survey system. As your team becomes familiar with it, you’ll develop your own methods. Consider putting together a best practice guide to share with your team.

That about covers it. And now we’d like to hand the mic over. From your experience, can you think of any other tips to increase software training productivity? Let us know. Just like you’ll learn from us, we’re sure there’s a whole lot to learn from you.

Let us help you.

Discover how we can help YOUR organization solve its current survey needs.