The Voxco Answers Anything Blog

Great questionnaires, discussion guides, and other types of survey data collection tools do far more than gather information to effectively answer research and business questions. They create a research environment in which every person participating feels safe and respected enough to share their truthful and complete opinion. It’s not quick or easy to develop such questionnaires but demonstrating to people that researchers truly care about their personal and customer experiences results in much higher data quality. And, it’s the right thing to do. Let’s consider three ways that social and market researchers can strive to achieve inclusivity through questionnaires. 

Cultural and ethnic inclusiveness 

It’s simple and obvious to say that racist language must never be used in questionnaires or any type of research data collection tool, but this can be difficult to accomplish. Language develops over decades and centuries, and reflects evolving political, cultural, and social perspectives. Our everyday language includes words and phrases that many people don’t realize have racist or sexist histories. For those writing questionnaires about home goods, meetings, or technology, here are just a few phrases with negative histories that need to be replaced. 

• Replace “master bedroom” with “main bedroom”  

• Replace “brown bag sessions” with “lunch and learn” 

• Replace “black list” with “block list” 

• Replace “powwow” with “meeting” 

• Replace “black sheep” with “outsider” 

• Replace “Illegal people” with “non-citizens” 

 As with the law, ignorance is no excuse. It’s not enough to remove language that is obviously racist or sexist. Researchers with high ethical standards actively seek to identify racist language and remove it from their vocabularies and their questionnaires. A quick Google search will lead you to many resources offering respectful alternatives. And, try uploading your questionnaire to an AI tool to catch any additional racist phrases you missed.   Similarly, take care to ensure that questions related to demographic characteristics do not incorporate stereotypical or non-inclusive phrasing, especially if modern phrasings do not mirror your own personal beliefs. For example: 

• Replace binary gender options (male/female) with multi-category options that accommodate people who are non-binary. 

• Replace stereotypical roles such as housewives, stay-at-home mom, fireman, and policeman with inclusive words like caregiver, homemaker, firefighter, and police officer. 

 To learn more about creating inclusive demographic questions, review some of these expert resources: 

• ESOMAR: Best Practice Recommendations for Measuring International Demographics 

• ESOMAR: Diversity, Equity, and Inclusion 

• Insights Association: Guide to: Inclusion, Diversity, Equity, and Access Council Toolkit 

• Oregon Health & Science University: Inclusive Language Guide 

 H2: Disability inclusiveness Today’s technology makes it vastly easier for non-experts to include people who are disabled. Adaptive keyboards, trackballs, head-tracking devices, eye-tracking devices, screen-readers, screen magnifiers, captioning software, and so many other tools mean that people who are disabled have more opportunities to access questionnaires without third-party assistance. However, researchers still need to actively ensure questionnaires are as accessible as possible.  First, ignore the thousands of cute and fancy fonts. Stick with simple, clean, and consistent fonts like Arial, Verdana, Helvetica or Calibri, fonts that are much easier for accessibility devices to display and decipher accurately. Avoid emojis, emoticons, ascii art, and fonts that incorporate imagery or shapes as these unnecessary ‘letters’ will be described in several words, thereby interfering with the logical flow of a phrase or sentence. Boring can be a good thing.  As we’ve already discussed, language matters. Researchers need to ensure questionnaires use people-first language. Rather than referring to “disabled people,” it’s more respectful to refer to “people who have a disability.” Similarly, replace negative attributions like “Are you wheelchair-bound” with neutral choices like “Do you use a mobility device?” There will always be a few exceptions (such as ‘autistic people’) so make sure to get advice from an expert within the intended population.  To learn more about creating documents that are more accessible to people with disabilities, click through to these resources: 

• National Center of Deafblindness: Accessibility Toolkit for State Deafblind Projects 

• AbilityNet: Creating Accessible Documents 

Language Inclusiveness 

Finally, language can be a massive barrier to the accessibility of questionnaires. Whether someone is fluent in six languages and is midway through their seventh or if someone knows only English and has a lower skill level, questionnaires must be written to accommodate as many people in the target audience as possible. Without language inclusiveness, only people who have at least the same level of fluency as the researcher will be able to participate in research. The end result is biased responses and potentially wrong research conclusions and business recommendations. What does this mean for questionnaire writers? 

• No big words: You finished high school and college. Perhaps you also earned a Master’s degree or a PhD. Your reading skills are much different than the majority of people. As such, you are a biased judge of what words and language most people will understand. The average adult in the USA reads at a grade 9 level. Write for them, not for you. 

• No jargon: Jargon isn’t just technical terms that doctors or engineers or factory technicians understand. Jargon includes words like “product category,” “touchpoint,” “pain point,” “targeting,” “user-generated content,” and “gamification.” Just because you see these kinds of words every day doesn’t mean they are part of everyone else’s vocabulary. Say what you mean, not what you report to your manager. 

• No Idioms: Like jargon, idioms are meaningless to people whose culture is different than yours. Avoid phrases like “think outside the box,” “raise the bar,” “go the extra mile,” or “push the envelope.” Instead, ask people to think carefully and be creative. Regardless of language skills, most people will understand these words. Once again, boring can be a good thing. 

Summary 

As researchers, we care deeply about data quality and completion rates. We are desperate to improve data quality so that our results are more valid and reliable. We’re desperate to increase questionnaire completion rates so our results are more generalizable to the intended population. Fortunately, by implementing techniques that make our questionnaires more accessible and inclusive, we accomplish multiple goals. It’s a win-win-win situation. With language and formatting that show people we genuinely care about them, we can improve research experiences in multiple ways. If you’d like to learn more techniques for creating inclusive questionnaires, please get in touch with one of our survey experts. We can help you build a better survey! 

Every researcher struggles with “the why.” WHY do people behave in a certain way. WHY do people prefer products of a certain shape, size, or color. WHY do people prefer a certain brand. These types of “why” questions often lead researchers to discussions about whether a qualitative or quantitative method is the best option for a particular market research project.

But “why” is not a question reserved for making a decision about using qualitative or quantitative research. Rather, it’s a question that social and market researchers should ask of themselves throughout the entire research process.

Why is your chosen methodology the most suitable methodology?

Researchers who specialize in questionnaire research are extremely skilled at their craft. In fact, they can probably answer any business question you bring to them by designing a comprehensive and detailed questionnaire. Similarly, skilled qualitative researchers can also solve a huge range of business problems with focus groups and IDIs. We know, however, that not every research question is best solved by using your favorite methodology.

• Baseline metrics: Are you seeking baseline metrics of frequency, magnitude, and duration? In such cases, quantitative research is your best option. While most people immediately turn to questionnaires as the best option, there are indeed many other quantitative options. Biometric methods like eye-tracking, EEGs, galvanic skin response, and heart rate variability offer valid and reliable metrics. Similarly, customer data analysis, web analytics and data mining could be more appropriate quantitative methods. Know WHY you chose your specific quantitative method.
• Generalizability: Are you trying to generalize behaviors or emotions from a small group of people to a vastly larger population? For this purpose, you’ll need to start with a fairly large random sample that is representative of the population in terms of key demographics and psychographics. Historically, quantitative methods were the only option but AI innovations have changed that. Today, qualitative research can be conducted at a vastly larger scale as tools like Ascribe can code and analyze vast quantities of qualitative data with high levels of accuracy.

There used to be a clear separation between what quantitative and qualitative research is and what it can do but that is no longer the case. Researchers need to ask themselves why they’re resorting to traditional methodologies when new options are being added to our toolbox every day.

Why are the questions so monotonous?

The bulk of marketing research data collection tools rely on asking people questions. Where do you shop? What do you buy? When do you buy? How many do you buy? Why do you buy? These are straightforward, simple questions. But, reducing our lives to these simple questions negates the complexity of our lives and our decision-making processes. Simple is good, but simple questions elicit shallow, habitual answers rather than deep, personal reflections.

Instead, we need to ask these questions: Why did I phrase the question like that? Why did I choose that set of answers? Why do I always rely on templated questions? Why do all of my questionnaires use the same questions? When we scrutinize our questionnaires in the same way we expect participants to scrutinize their answers, the result is more effective questions.

Simple questions don’t need to be mind-numbingly boring and lacking in complexity to generate thought-out answers. Choose the five most important questions in each questionnaire and take thirty minutes to brainstorm ten alternatives of each. Write out as many preposterous questions as you can. Imagine unusual scenarios, unexpected shopping partners, and unlikely shopping occasions. Aim for strange and unexpected. Even if none of the resulting questions are good, the process will push you towards questions that are more interesting and thought-provoking for your participants.

Why did you choose that type of report?

The fastest way to finish a report is to write it in PPT or Word. It’s what we’ve always done and it’s what clients have always expected. But, there is no reason besides convention that findings need to be shared that way. The purpose of a report is to share or teach a key learning to people, not to create pages of writing. So why did you choose PPT or Word?

Think about what you love to do in your spare time. Maybe you do like to read. In that case, Word might be the perfect type of report for you to receive. Even better, imagine if it took on the flavor of your favorite author and was written as a romance, mystery, or historical fiction novel. That might be the most engaging book you’ll even read and it will certainly be one you’ll remember forever.

Obviously, the creative report needs to be accompanied by an addendum of detailed results but there’s no reason for the most important teaching tool to be prose.

What’s next?

An extremely effective way to achieve business success is to reject the status quo. Instead, ask yourself why. Why did I choose this – because you did in fact choose every aspect of the research process, even if you did it without thinking. Follow up that why with multiple probes until you know for sure that you’re choosing the best path, not the easiest, fastest, or simplest path.

For quantitative researchers, it could mean recommending qualitative interviews with AI coding and analysis for your next project. For questionnaire authors, it could mean rejecting your traditional template and developing a new template replete with creative options that inspire deep thinking. No matter what your why is, it’s sure to put you on the path to more effective and engaging research. We’d love to be on that path so please get in touch with one of our survey experts!

This year, forget New Year’s resolutions. Instead, capitalize on your growth mindset and choose goals that fit naturally into your career path. Whether you’re seeking personal growth or business growth, here are a few ideas to get you started!

Get Comfortable with AI

Like it or not, AI is here to stay and it’s changing everything. In the insights industry, it’s infiltrated recruitment, sampling, research design, questionnaire design, interview moderation, data analysis, reporting, and more. There is no avoiding AI.

If you want to remain relevant and happily employed, you have no choice but to engage with and become knowledgeable about AI. You don’t need to become an expert programmer or developer but do need to be able to engage in meaningful conversations and make wise decisions. Here are some fantastic free and paid resources to get you started.

• Coursera: One of my favorite free sources for learning and improving skills, Coursera offers myriad free and paid courses from many accredited institutions you already know and trust. Perfect for beginners, you can sign up for their free online class called, “AI for Everyone” taught by Andrew Ng from Stanford University. This class focuses on understanding the terminology, what to expect from AI, how to use AI, and how to use AI ethically. You don’t need any technology skills or experience to benefit from this course.
• Harvard University: If you want to name-drop and you already know Python, this course is for you. Harvard offers many free courses to the public including an AI class called: “CS50’s Introduction to Artificial Intelligence with Python.” You’ll learn about algorithms, machine learning, AI principles, and how to use AI in Python. Make sure to drop this certification on your LinkedIn page!
• NewMR: If you’d prefer to be more of an observer and soak up tidbits along the way, sign up to receive Ray Poynter’s AI Newsletter. With lots of discussion about how to use AI and notices about webinars and new tools, Ray will keep you up to date with everything insights experts should be aware of. This is a great option for people who feel they’re too old or too far along in their career path to start learning something new – because Ray isn’t!

Increase Your Questionnaire Design Skills

No matter how many years of experience you have writing questionnaires, there’s always someone from a different industry with different experiences who can offer you something new to learn. Whether that’s new types of questions or perspectives you hadn’t considered before, grow your questionnaire design skills with a free or paid class, webinar, or book.

• ESOMAR Academy: Always a wealth of current knowledge, ESOMAR regularly offers online classes including this “Empathetic Survey Design” course by Jon Puleston and Martha Espley. Stuffed with data from their own research, you’re sure to pick up some helpful techniques you’ve not considered before.
• Coursera: The choices on Coursera are unending. Whether you prefer the perspective of psychologists, sociologists, anthropologists, or economists, there is a questionnaire design course for you. For a well-rounded approach, try this “Questionnaire Design for Social Surveys” course offered by the University of Michigan.
• People Aren’t Robots: Skip this option if you don’t like authors plugging their own books! In case you do, “People Aren’t Robots” is a short yet detailed book with lots of examples that will inspire you to write better questionnaires. Taking the point of view that people are imperfect, it offers a rare perspective on how to be kinder to people answering questionnaires.

Be the Change You Want to See

It’s easy to get frustrated when you see your industry moving down one path when you want to take a different path. Fortunately, there’s an easy solution. Join your national association and grab the steering wheel!

• Insights Association: If you’re based in the USA, the Insights Association works on your behalf to champion, protect, and create demand for the insights and analytics industry. Volunteers participate in developing quality standards, education, certification, and more. Get involved with a few small projects to see just how big of an impact you can make.
• CAIP and CRIC: If you’re based in Canada, you’ve got two ways to help steer your national association. Individuals can become Certified Analytics and Insights Professionals and companies can join the Canadian Research Insights Council. In either case, volunteer as a board or committee member, and make your priorities come to life.
• Esomar: Are you ready to take a bold move and create global impact? Then Esomar is your destination! You may feel you’re not yet ready to run for a board position or a National Representative position but there are plenty of other ways to be heard. That might be as part of project committees that work towards specific tasks like guidelines and publications, or program committees that plan events and content. Contact Esomar to find out how you can make your ideas a reality.

What Next?

As the saying goes, the best time to start is right now. Whether it’s January 1st or June 30th, exercise your mind with a course, a webinar, a book, or an email to your national association. And when you’re ready to implement your upgraded questionnaire design skills or test out an AI text analytics and verbatim coding system, talk with one of our survey and AI experts. We’d love to be a part of your growth journey!

No matter how careful you are, false precision creeps into research results and reporting. From research design and statistical testing, to a general failure to accept the limitations of people and data, numbers often appear more accurate than they really are. There are a few things researchers can do, however, to try to minimize false precision such that marketers can make more informed business decisions.

Incorporate as much rigor as possible

Researchers have many tools to reduce the potential for false precision but three foundational techniques are particularly important.

First, use the largest sample sizes you can afford. While there is no “best” sample size that applies to every study, it’s fair to say more is better. In the market research space, 700 per group often offers the precision necessary to determine whether two groups are different - in this case, 10% vs 15% will probably be statistically different. When budgets lead to sample sizes of 200 to 300 per group, reliability will decrease and false precision will increase.

Second, use comparison or control groups as often as possible. Without a comparison, it’s impossible to know how much random chance affected the data. Was recall of your brand actually 10% or would 10% of people have recalled a brand you just made up? Did 10% of people try or buy or like or recommend your product or would 10% of people have said the same of a brand you just made up? No matter how careful they are, people will always misremember and misunderstand seemingly obvious things.

Third, when the opportunity arises, use a true random sample. If you’re lucky enough to be working with students registered at a school or cashiers employed in a store, it may be possible to gain consent from a sample of the population. Unfortunately, most market researchers won’t have access to a population list of customers/shoppers/buyers/users and so won’t be able to benefit from this.

Use as few significant digits as possible

Numbers are easy to generate. Throw a questionnaire at 700 people, run chi-squares, calculate pvalues, and build thousand-page tabulations. But those resulting numbers aren’t truth. They are representations of complex subjective constructs based on fallible, unreliable humans. Where truth is 68%, a survey result could be 61% or 69%. To say that 61.37% of people would recommend hypothetical Brand C is a gross misuse of decimal places.

Decimal places are perhaps the most problematic source of false precision, particularly in the marketing research world. To avoid this, don’t use any decimal places when percentage values are between 5% and 95%. Similarly, avoid using two decimal places when reporting Likert results. Only venture into one or more decimal places when you’re working with huge sample sizes from truly random samples.

Even better, if you’re brave and want to express your appreciation for false precision, round 61.37% to ‘about 60%.’

Use statistical testing wisely

Like artificial intelligence, statistical tests are meaningless when they aren’t offered with human oversight.

Tabulation reports can include thousands of t-tests and chi-square tests but, by design, we know that 5% of the significant results are Type I errors. Even worse, we don’t know which of those significant results are false. Because they are easy to find and exciting to report, it’s easy to overuse these significant results. To help readers grasp the concept of false precision, it’s a good idea to share corroborating trends from other sources such as last year’s research report, loyalty data, economic or political data.

If you’re lucky enough to be using random samples, always report margins of error. Further, always report available confidence intervals. While these numbers also incorporate a degree of false precision, readers need reminders that any statistics shared aren’t carved in stone.

Most importantly, ensure your reader understands that any numbers presented are not truth. Rather, they are vastly closer to truth than hypothesizing.

Summary

False precision is an easy trap to fall into, especially when the research results match your hypotheses. It can result in misleading interpretations, flawed decision-making, and ultimately, negative consequences for businesses. However, by being mindful of the limitations of research designs and data reporting, and offering clear instructions on how to best interpret numbers, researchers can help marketers better understand their data and make more informed and accurate decisions. If you’re curious about false precision might present itself in your research, feel free to connect with one of our survey experts!

Companies are in a heated race to showcase how their AI innovations help businesses achieve significant gains in processing data more quickly and accurately. But, as with all things, jumping on the bandwagon without a plan is rarely a wise move.

As AI tools developed in a variety of industries over the years, their use has uncovered lessons for future innovators. Here are just a few of those lessons.

Sudden pivots are wise

Though it seems like a lifetime ago, it was only eight years ago that Microsoft released the Tay chatbot. It instantly became a playful conversational tool with cute little quirks. However, people quickly migrated from having fun conversations with it to engaging in more controversial use cases. They realized they could train Tay, and soon taught it to become racist, sexist, and hateful.

This unanticipated outcome led to two important learnings. First, Microsoft reacted quickly and removed Tay from public use. Tay’s removal did not necessarily reflect a failure but, rather, a calculated risk within the innovation funnel. Second, as we’ve already learned from “Privacy by design” methods, the Tay incident reinforced the need for AI tools to incorporate “Ethics by design” models. Thanks in part to Tay, most AI tools now incorporate ethical guardrails. Take the risks, bring your innovations to market, but ensure they are prebuilt with processes to detect and prevent misuse.

Minimum viable standards are relative

Remember when restroom hand dryers with sensors first came out? They worked great for many people, but it soon became apparent that they were unreliable for people with darker skin tones. Developers hadn’t tested the product on people who had darker skin. Across the board, we’ve seen that AI tools are often biased towards pale, male faces because other people are excluded in sufficient quantities from AI training data. As a result, we now have higher minimum standards for training datasets, and we ensure they include people reflecting a wide range of demographics, especially in social and market research.

Our standards have improved over time, but they also differ based on the use case. In the research industry, for example, if you need to code questionnaire verbatims to understand which color of bar soap people prefer, 85% accuracy is suitable for the job. Increasing the 85% to 95% won’t change the outcome of the research but it will take longer and cost more. On the other hand, if you need to understand the efficacy of different mental health therapies, achieving 99% accuracy via automated coding enhanced with manual coding is the better way to go. Life-and- death situations necessitate higher accuracy. Standards are relative.

Ensure people retain final oversight

If you ask several competitive AI image generation tools to create an image of fish in a river, and they all show sushi and maki rolls swimming upstream, that doesn’t make the image valid. In fact, after seeing just one image, people would know the result was invalid.

This is exactly why people are necessary to confirm the validity and accuracy of AI tools. For example, during the development of our qualitative coding tool, Ascribe, we compared the results generated by the AI tool with results generated by expert human coders. It takes time to continually generate results in various industries and topics, and then test those results with human coding. But, that ongoing process is time well-spent to ensure that the quality of AI results is comparable to or better than human results.

Cautious risk-taking will win the AI race

Perfection is elusive, even in the age of AI. Every product, no matter how advanced, has its limitations. While some flaws, like those seen with Tay, might demand drastic changes, most can be addressed with small tweaks or by discovering the optimal use cases. The secret to successful innovation is a balance of bold ideas, agile adaptation, and the courage to take small, calculated risks. If you’re curious to learn more about our approach to using AI wisely, please get in touch with one of our Ascribe or Artificial Intelligence experts.

Research reports come in a full range of formats. From short and precise to long and detailed, there’s a report for that. In this post, we’ll outline the pros and cons of three formats to help you choose the one that will best suit your business needs.

Data reports are cheap, fast, and easy

Historically, the research industry has done a great job of producing reports with reams of data. Today, these types of reports are supported by automated systems that produce a nicely colored chart for every single question in a questionnaire. Supplemental charts might also be built manually using data pulled from tabulations. With charts in place, titles or headlines are easily prepared by converting the largest or smallest number in each chart to a phrase.

You know you’re working with a data report when the headlines read like these:

• 75% of shoppers seek out discounts
• 55% of people prefer package B
• 80% of people indicated child-care is a key issue

Data reports are fast and relatively cheap to create. Launch a survey, request the automated charts, and choose an interesting number to become the title on each slide. These reports require little thinking and creativity and, as such, nearly anyone can prepare them regardless of their skill or experience.

Despite their tendency to be quite long, these types of reports answer limited questions. And though automation means that they’re far less expensive, they are the least helpful and are rarely used again. To improve data visualization and streamline reporting, consider integrating dashboards using modern admin panel themes, a quick solution that provides interactive and user-friendly insights into your findings.

Insight reports take time, money, and care

In recent years, our industry has realized that data reports offer little value to research buyers. “What” without the “how, why, and when” doesn’t lead to repeatable business decisions that can be generalized for long-term value across products and audiences. Consequently, many report providers have turned their efforts from data reports to insight reports.

Insight reports require researchers to simultaneously consider multiple pieces of data and derive an understanding that isn’t inherently obvious in the data. It requires understanding the external, environmental context in which the research took place, and how this connects to the business problem at hand.

Insight reports are more expensive and take more time to create because they require more skill and experience to create. Rather than focusing on the largest or smallest number in a chart, researchers instead seek out the unexpected, nonsensical, out of place numbers which are most important. Turning those numbers into insight requires digging into external data – cultural, historical, geographical, political, technological, economic.

Headlines in an insights might look like this:

• Because of the sudden cost of living increase, 50% more shoppers are value seekers
• Despite being less accessible, people preferred Format B because their favorite celebrity was recently seen using it
• The need for child-care declined to 80% due to higher unemployment rates

Although insight reports take more time and money to create, they create value beyond the cost of data collection. They offer insights and understanding that can be used not only for the business problems at hand, but also for other challenges still to come with similar products, categories, or audiences. In the end, these more valuable insights lead to long-term loyalty between the researcher buyer and provider.

Action reports are expensive but they generate ROI

The most valuable reports are action reports. These reports go beyond data points and contextual insights to offer actionable recommendations that identify long-term solutions for specific and generalized business problems. If you were to map out the questions and slides in these types of reports, you wouldn’t find a chart for every question and the slides wouldn’t be ‘in order.’ Rather, you would find that each slide title contributes to an overall coherent story complete with an introduction, key issues, and final conclusions.

Here are some examples of action statements in a report

• With the increased cost of living creating more value seekers, switching to lower quality paper and using less ink on each printing will help decrease prices by 8%
• Despite lower ratings for Format A, it should be adopted with a marketing plan that demonstrates these 3 accessibility use cases
• Though demand for child-care has decreased, we must increase child-care openings to support job-seeking opportunities

Unlike lengthy and less expensive data reports, action reports are usually short and much more expensive mainly because they are vastly more difficult to create. Action reports depend on a true partnership between the research buyer and provider, a partnership wherein the provider has been privy to the confidential workings and challenges experienced by the research buyer. A true partnership leads to reports that offer long-term value and increasing ROI.

Key Takeaways

The choice between data reports, insight reports, and action reports ultimately boils down to your specific business needs. While data reports offer a quick and cost-effective solution, their limited insights may provide no actionable recommendations. Insight reports, on the other hand, dive deeper into the data, uncovering valuable patterns and trends. And, at their best, action reports truly deliver by providing not only insights but also concrete recommendations to drive business growth. The key to success lies in choosing the type of report that aligns with your strategic goals and provides the actionable insights you need to make informed decisions. If you’d like to understand more about research processes, connect with one of our survey experts. We always love chatting about the best ways to uncover valid and reliable insights!

Market researchers and data analysts are constantly looking for faster and more efficient ways to analyze and extract actionable insights from vast amounts of open-end feedback. The launch of Theme Extractor 2.0 and Ask Ascribe marks a major leap forward in open-end analysis, equipping organizations with advanced tools that offer exceptional speed, precision, and depth of insight.

Let’s explore how these innovative solutions enhance Coder, Ascribe’s verbatim coding platform, and CX Inspector, Ascribe’s text analytics solution, along with how they can improve your workflow.

What is Theme Extractor 2.0?

Theme Extractor 2.0 is Ascribe’s latest AI-powered innovation, designed to automatically analyze open-ended comments and survey responses with an accuracy of over 95%. It generates a human-like, richly descriptive codebook with well-structured nets, allowing users to spend less time on the technical aspects of analyzing open ends and more on interpreting results. By using Artificial Intelligence (AI) and Natural Language Processing (NLP), the tool navigates the complexities of text analysis, dramatically reducing manual analysis and delivering results with incredible speed.

Check out the theme-based codes and nets in the following snapshot of results, delivering a significant improvement over the one-word topics used in older technologies. A total of 1655 responses were analyzed, and 1638, or 99%, were immediately classified into 33 codes and 9 nets.

Benefits of Theme Extractor 2.0

1. Accelerated Workflow and Efficiency

Analyzing open-ended responses manually can be extremely time-consuming. Theme Extractor 2.0 automates this task, allowing users to process large datasets and free up valuable time for more in-depth analysis. With minimal manual intervention required, it streamlines your workflow and reduces the risk of errors.

2. Exceptional Accuracy

With over 95% accuracy, Theme Extractor 2.0 provides a level of precision that rivals human coding. By minimizing uncoded responses and overlapping codes, it ensures that your data is clean, structured, and primed for thorough analysis. You can trust that the data and insights are as accurate and reliable as possible.

3. Clear and Organized Codebook Structure

Theme Extractor 2.0 generates theme-based codes and nets that help uncover deeper patterns within your open-end responses. Say goodbye to one-word topics, and hello to helpful, descriptive results. It structures codes into clear, logical nets, making it easier for researchers to interpret data and spot key themes. These improved results add clarity to your analysis and ensure a seamless experience.

4. Minimal Manual Intervention

With minimal manual intervention needed, Theme Extractor 2.0 handles the labor-intensive tasks, allowing users to focus on more strategic work. Automating analysis allows researchers to quickly move from raw data to actionable results, saving time and reducing the risk of human error.

What is Ask Ascribe?

Ask Ascribe is an innovative AI-powered tool that lets you ask natural language questions about your data and get instant, actionable insights through answers, reports, and summaries. Powered by advanced Generative AI models, Ask Ascribe allows researchers to engage directly with their data, enabling them to ask specific questions and gain a deeper understanding of key themes, customer sentiments, and areas for improvement.

What is Ask Ascribe?

1. Instant Answers with Natural Language Interaction

You don’t have to sift through piles of data anymore—Ask Ascribe lets you ask natural language questions like “What are the main themes in this feedback?” or “How can I improve my Net Promoter Score (NPS)?” and receive accurate, data-driven answers in seconds. This AIpowered feature simplifies the analysis process, providing you with real-time insights into the story behind your data.

2. Uncover Actionable Insights Easily

Ask Ascribe goes beyond providing answers; it delivers actionable insights that empower you to make informed decisions. Whether you’re looking to explore customer emotions, identify pain points, or uncover opportunities for improvement, Ask Ascribe helps you quickly grasp what your data is revealing and what steps to take next.

3. Explore Data on a Deeper Level

With Ask Ascribe, you can easily drill down into responses to see the original comments tied to specific insights. This feature allows you to explore your data more thoroughly and gain a nuanced understanding of key drivers, customer feedback, and other qualitative insights. You’re not just scratching the surface—you’re engaging deeply with your data.

4. Empowers Better Decision-Making

By delivering quick, relevant answers to your questions, Ask Ascribe helps you make informed decisions based on real data. The ability to “interview” your data in real-time empowers researchers, analysts, and business leaders to respond faster and more strategically to emerging trends and customer feedback.

Why Theme Extractor 2.0 and Ask Ascribe Matter

Together, Theme Extractor 2.0 and Ask Ascribe – available as features in Ascribe’s Coder verbatim coding platform, CX Inspector text analytics and Services solutions – are changing how researchers and analysts engage with their data. By automating open-end analysis and coding processes and enabling AI-powered Q&A, these tools drastically reduce the time and effort needed to turn open-ended responses into clear results and actionable insights. Here’s why they make a difference:

• Unparalleled Speed to Results: Load up a data file or ask a question and the results appear within seconds. Now, any dataset with open-end responses of any size and complexity can be analyzed quickly and easily.
• Increased Productivity: These solutions streamline workflows by eliminating manual processes, enabling researchers to analyze more data in less time.
• Improved Accuracy: Both tools leverage advanced AI models to deliver precise, more descriptive results, ensuring you can trust the insights derived from your data.
• Deeper Understanding of Data: By offering theme-based codes and nets, and natural language interaction, researchers can dig deeper into their data, leading to more strategic decision-making.

Conclusion

The ability to transform open-ended data into clear, actionable results is essential for any organization that values customer feedback and market research. Ascribe’s Coder, CX Inspector, and Services, available with Theme Extractor 2.0 and Ask Ascribe, are powerful solutions that deliver a new level of speed, accuracy, and ease to researchers and analysts.

These solutions are the future of open-end data analysis—enabling users to unlock valuable insights with minimal effort and maximum efficiency. If you’re ready to transform how you work with data, it’s time to explore what Ascribe can do for you.

Interested in discussing how Ascribe can help you?Click here to sign up for a live demo with your data,or drop us your contact info and we will reach out to you.

Click here

Whether your environment is academic or industry, every social, market, and consumer insights expert has developed a unique set of skills shaped by their education and experience. Regardless of the depth or source of these skills, they remain, however, incomplete. With that in mind, here are some key skills that industry and academic researchers can learn from one another.

Embrace the randomness of real life

Every researcher loves a fully factorial, experimentally designed retail aisle, but those don’t exist in real life. Real-world shopping means not finding a product, brand, size, or format you’re looking for, coming upon unexpectedly low or high prices, and dealing with rude customers and employees.

Practical researchers who conduct on-site shop-alongs and ethnographies have extensive experience analyzing and interpreting complex, messy, real- life scenarios. Their hands-on experience makes their work highly relevant to business leaders needing to understand current industry needs, market trends, and consumer preferences. Real life may be messy, but academic researchers should learn to embrace a bit more mess.

Incorporate more theoretical depth

Human behavior isn’t new. For more than a hundred years, academic researchers have worked to understand market and consumer behaviors and to build theoretical foundations like Cognitive Dissonance Theory and the Diffusion of Innovation Theory that can be used to hypothesize and predict future behaviors. They’ve built on the work that came before them, knowing that it is the foundation of their own research. This is what elevates their work from simpler descriptive analyses and hypotheses to deeper understandings of who, what, when, where, why, and how certain consumer behaviors occur.

Rather than trying to understand a single problem, such as which package will be successful today, academic researchers work to build theories that will have a broader impact. By digging into the human behavior archives more often, industry researchers could also generate more robust conclusions and recommendations.

Practice agile research processes

Where academic researchers often have months or years to run a research project that accommodates a wide range of variables, industry researchers often have days or weeks. Industry researchers have learned to expect and adapt to changing circumstances so that they can meet rapid turnaround times. Their work is efficient and responsive to the real world, which can change literally overnight. Industry research is often simple and quickly actionable. Bang for the buck is clear and personally observable. Academic researchers could definitely benefit from tightening their timelines and getting their outcomes into the real world more quickly.

Engage More Stakeholders

Determining whether customers like package A or package B more is not as simple as it seems. Yes, a highly controlled experiment will reveal a winner, but simply moving forward with the results of what is essentially a customer vote could lead to a massive failure.

To ensure an experiment doesn’t land in the proverbial file drawer, industry researchers are careful to engage many of their stakeholders, including not only customers but also the package designer who will need to make any subsequent tweaks to their beloved design, the brand manager who campaigned for the losing package, the category manager who must budget for a more expensive package, the business development team who must promote a design they don’t personally like, and the executive leadership who is divided on the decision.

By engaging a cross-disciplinary team from the beginning, industry researchers have learned how to strengthen the applicability and reach of their research. Sometimes, academic researchers need to remember that uncovering truth isn’t the automatic path to success.

Take risks with new innovations

If you’re not already using AI, your competitors will pull ahead and leave you behind. That doesn’t mean you should jump onto the AI bandwagon and use it everywhere you possibly can. It does mean you should find ways to incorporate new methodologies like AI as soon as you find practical and valid uses for them.

Rather than waiting for academic researchers to complete highly controlled studies, industry researchers are incorporating AI tools and techniques for experimentation along the way. With side-by-side comparisons, industry researchers allow their customers to see and get comfortable with the results while also ensuring the innovations are valid and reliable in real life. The key is to take safe and considered risks along the way.

Value long-term learning

You’re a researcher because throughout your life, as a baby, toddler, child, teenager, and young adult, you were exposed to a set of experiences and circumstances that consciously or unconsciously led you to reject some job opportunities and choose this one. You are a lifelong experiment, not a one- time study from last week.

Similarly, academic researchers value lifelong experiences, which means that many of them conduct studies that take 5, 10, or even 50 years to complete. These studies help us understand systemic, longitudinal issues that are not visible in one-time, cross-sectional studies. Longitudinal studies are how we understand the impacts of early education on later voting patterns or early community service experiences on later-life consumer issues. Industry researchers would be well-served to take a longer-term approach to some of their studies.

Incorporate more scientific rigor

Academic researchers are rigorously trained in statistics, sampling, and research methodologies. Through years of schooling, they’ve learned to scrutinize and interpret data with a critical eye. They know when to use a Scheffé correction and why they won’t use it this time. They know the pitfalls of data tabulations and when to use Python and R scripts. Consequently, they succeed with high research quality and validity, minimal biases, and maximum reliability.

Given that many market and consumer researchers are serendipitous members of the industry and only receive on-the-job or as-needed training, this is definitely a gap that needs to be filled.

Summary

Whether you’re an academic or industry researcher, we all have knowledge gaps across a range of areas. Having a growth mindset focused on uncovering and filling those gaps is how good researchers eventually become great researchers. We would all benefit from taking a masterclass from ESOMAR or a course with the MRII, so don’t be shy. Have a look at their offerings and see how you can add new skills to your repertoire. Or, if you’d like to chat with a passionate colleague about research techniques, please connect with one of our survey experts. We’d love to hear from you!

Introduction

As the political landscape shifts beneath our feet, polling remains a critical tool for understanding public sentiment. What if the path to more accurate voter insights lies in learning from our past experiences?

To deepen our understanding, we had a conversation with Dr. Don Levy, Director of the Siena College Research Institute, who shared valuable perspectives from his extensive experience in polling. His insights, derived from our discussion as well as his appearances on podcasts with AAPOR and WXXI News, shed light on the factors that can skew polling results.

In this blog, we’ll explore key lessons from past elections and the innovative strategies pollsters like Dr. Levy are implementing to enhance their methodologies and restore public trust.

Key Lessons from the 2016 and 2020 Elections

The polling industry has learned vital lessons that have reshaped its approach to electoral forecasting. Dr. Don Levy discusses the factors contributing to bias observed in the last two elections:

“I think we all learned a great deal from 2016, which, at this point, feels like the distant past. There was an insufficient amount of state polling during that election, and some battleground states—like Michigan, Wisconsin, and, to some degree, Pennsylvania—were not even identified as such. Additionally, some polls did not weight responses by education in that cycle. We learned from those mistakes.

By 2020, the polling landscape had changed entirely. Everyone was considering education, recognizing it as one of the key fissures in the American electorate, and there was a greater focus on several key battleground states. We were not alone in polling Wisconsin, Michigan, and Pennsylvania at that point.”

Dr. Levy identifies two critical areas for improvement:

1. State-Level Polling:
   Many national polls failed to accurately gauge support for candidates due to an insufficient focus on state-level data. This oversight revealed how local dynamics and voter sentiment vary significantly across different regions, leading to inaccuracies in predictions.
2. Education as a Variable:
   The impact of education on voter preferences was underestimated in many polls. Dr. Levy notes, "Education proved to be a dividing line, especially in battleground states." This insight emphasizes the importance of moving beyond traditional demographics. Pollsters must now consider how educational background influences voter behavior to enhance the accuracy of their models.

‍

In response to these lessons, methodologies have evolved to prioritize regional polling and implement nuanced weighting practices that better represent educational backgrounds. By learning from past missteps, pollsters are striving to produce more reliable results, thereby reinforcing the credibility of polling as a tool for democratic engagement.

This recognition of education as a crucial variable, alongside the emphasis on state-level insights, marks a significant shift in voter sampling approaches. The goal now is to capture a clearer picture of public sentiment, ensuring that polling remains a trusted resource in the ever-changing political landscape.

Addressing Challenges with Innovative Approaches

In response to recent election cycles, pollsters have made significant strides to tackle key challenges. Dr. Levy highlights how these adjustments contribute to better accuracy and engagement:

• Achieving Diverse Representation Through Stratified Sampling: Pollsters use stratified sampling to ensure diverse representation across demographics, selecting sample groups that mirror the broader population—not only by age, gender, and race but also by considering regional and socioeconomic nuances. This helps create a more accurate model of voter intentions.

“We faced a bias in 2020. Our view was that ardent Trump voters—not the shy ones—tended not to respond to polls. This issue was evident across all polling methods: phone, web-based, text-to-web, and IVR. In analyzing about 50 polls from reputable AAPOR members in the ten days before the election, only three correctly predicted outcomes or leaned Republican, while nearly all others were biased the other way.

In Pennsylvania, for instance, areas with the highest Trump support showed significant polling errors. In regions where Trump won by 70% to 30%, polls often showed him at around 60% to 40%. But systemically, what occurred was that as we filled our quotas, we would just simply get a greater participation among Biden voters who were, to stereotype, white men without a college education than from a representative sample of Trump voters.

So, do we face that threat again this election? Yes. To address it, we’re implementing rigorous quota management, stratified sampling, and actively reducing drop-offs, along with the benefit of a repeat election.”

• Minimizing Drop-Offs by Leveraging Historical Data and Voter Lists: Pollsters have also become more vigilant about minimizing drop-offs, aiming to retain as many responses as possible to enhance data robustness. "Drop-offs can skew results by unintentionally filtering out certain groups," says Dr. Levy. As a solution, historical data and detailed voter lists have become valuable tools, allowing pollsters to refine their models and develop weighting strategies that are more reflective of the electorate. This approach helps account for variances in response rates, especially among underrepresented groups.
• Increasing Voter Engagement using AI and Data Enhancement: With advancements in artificial intelligence and data enhancement, pollsters now have more sophisticated tools to reach and engage voters, even as response rates remain challenging. AI-driven insights can optimize contact strategies, enabling pollsters to identify when, where, and how to engage different voter groups effectively.

These innovations collectively improve polling accuracy, reinforce public trust, and ensure that polling remains a valuable tool in capturing voter sentiment.

Upholding Integrity: The Importance of Non-Partisan Polling

In commitment to Non-Partisan Polling, maintaining public trust is paramount, and organizations like the American Association for Public Opinion Research (AAPOR) are essential in promoting transparency and upholding high standards in the polling industry. Dr. Levy highlights that these organizations work to enhance public understanding of non-partisan polling, which serves as a critical check against potential biases that can arise in politically charged environments.

“My hope—and my request—to my friends in the polling community is to broadcast to U.S. citizens and voters, explaining who we are, what we do, and why it’s important. It would be a positive act of citizenship to participate in high-quality, non-partisan political polls. We're not trying to sell you anything, convince you of anything, and/or manipulate you in any manner, shape or form.”

Pollsters are committed to contributing positively to the democratic process by fostering greater transparency around polling methods, data sources, and any limitations that may impact findings. Through collaboration and innovation, the polling industry continues to adapt and improve, addressing new challenges with integrity. As the political and social landscape evolves, the industry’s focus on high standards remains steadfast, ensuring that polling serves as an informative, unbiased reflection of public sentiment and helps advance informed democratic engagement.

Conclusion

Reflecting on the lessons of recent elections, it’s evident that the future of polling depends on how well the industry adapts and innovates. By embracing better sampling techniques and new technology, pollsters are working to paint a truer picture of where voters stand and what they care about. The path forward is one of continuous learning and evolution, ensuring that polling remains a trusted tool for understanding public sentiment.

Open-ended questions and analysis provide a rich layer of insights that goes beyond the limitations of structured data, offering a clearer understanding of customer sentiment, emerging trends, and overall experience. Whether gathered from surveys, customer reviews, or social media, this feedback can be a goldmine for business insight—but analyzing it effectively can be a challenge. For those dealing with thousands of open-ended questions and responses, traditional methods such as reading through open ends, tallying up responses in Excel, or simply not having the capacity or the right tools to analyze the feedback becomes overwhelming. Additionally, using generic AI tools like ChatGPT often proves inadequate. Without a qualified solution to easily analyze open-ended responses, researchers risk generating incomplete or inaccurate results, which can lead to delays and inefficiencies in completing projects.

At Voxco, one of the reasons we are excited about integrating Ascribe into our online survey platform is because of how Ascribe’s Coder and CX Inspector simplify and streamline open-end analysis, especially for corporate and market researchers. These tools leverage the latest advancements in artificial intelligence, making it easier to unlock actionable insights from vast amounts of open-ended data. This exciting product release showcases our joint commitment to offering innovative solutions that help researchers drive more impactful decisions.

Why Open-Ended Questions and Analysis Matter

• Deeper Customer Insights: Open-end analysis delivers a more comprehensive view of customer behaviors, motivations, and sentiments, helping companies better understand their audiences
• Uncover Hidden Insights: Open-ended responses can reveal unexpected insights that structured data misses, offering an early window into new trends and opportunities.
• Capture Nuances: Unlike structured responses, open-ended feedback allows customers to express their feelings and experiences in their own words, providing valuable context that helps businesses understand emotional drivers and pain points.
• Identify Emerging Trends: By analyzing open-ended feedback, businesses can spot new patterns that may not have been considered in the initial survey design, allowing them to respond quickly.
• Understand the ‘Why’: While quantitative data shows what’s happening, open-end feedback reveals why customers behave the way they do, offering more actionable insights.

How Ascribe’s Coder and CX Inspector Can Help

Ascribe’s industry-leading tools enable the analysis of open-ended feedback at scale with greater accuracy and speed by seamlessly blending artificial intelligence with human expertise. Here’s how Ascribe Coder and CX Inspector, both recently upgraded with Theme Extractor 2.0 and Ask Ascribe, empower researchers:

Theme Extractor 2.0

Automatically analyzes over 95% of open-ended responses and generates accurate, human-like codebooks. The tool eliminates overlap between themes and delivers cleaner, faster results—ideal for streamlining the research process.

Ask Ascribe

Ask Ascribe allows users to query their data in real-time using natural language. This interactive approach enables the quick identification of key themes, emotions, and areas for improvement, allowing businesses to act on insights faster.

Ascribe Coder: Enhancing Coding Productivity

Ascribe Coder enhances productivity by converting unstructured text into structured data. Here’s how:

• AI-Driven Human Coding: Combines AI with human intelligence to accelerate the coding process while maintaining precision.
• Customizable Automation: Allows users to adjust the level of automation based on project needs, ensuring control over cost, timing, and accuracy.
• Data Integration: Integrates open-ended feedback with survey data, providing a 360-degree view of the customer experience.

CX Inspector: Advanced Text Analytics for Deeper Insights

CX Inspector offers a robust platform for extracting and visualizing themes, sentiment, and emerging trends from open-end responses. Key features include:

• Instant Theme Extraction: Automatically identifies clear, descriptive themes from open-ended responses.
• Sentiment Analysis: Detects and visualizes customer sentiment instantly, enabling businesses to prioritize issues based on emotional impact.
• Actionable Insights: Combines theme detection and sentiment analysis to deliver clear, actionable insights that can be shared easily via dashboards and reports.

Conclusion: Unlock the Full Potential of Open-Ended Questions and Analysis

At Voxco, joining forces with Ascribe is part of our mission to empower researchers with powerful tools for analyzing open-ended feedback. As Rick Kieser, Chief Strategy Officer at Voxco, explains:

“Ascribe has dedicated 25 years listening to customer feedback and analyzing open-ended feedback, partnering with the world’s top market research firms and industry-leading corporations. By closely listening to the needs of these pioneers and continuously evolving, Ascribe has delivered cutting-edge solutions that shape the future of text analytics. The launch of Theme Extractor 2.0 and Ask Ascribe represents the pinnacle of this expertise—a culmination of decades of innovation, hard-earned insights, and the processing of over 6 billion customer comments. We’re excited to bring these solutions to Voxco’s customers and continue pushing the boundaries of innovation in research.”

With Ascribe Coder and CX Inspector, researchers can efficiently categorize and act on open-end feedback, driving more informed decisions and enhancing the overall customer experience.