The Voxco Answers Anything Blog

No matter how careful you are, false precision creeps into research results and reporting. From research design and statistical testing, to a general failure to accept the limitations of people and data, numbers often appear more accurate than they really are. There are a few things researchers can do, however, to try to minimize false precision such that marketers can make more informed business decisions.

Incorporate as much rigor as possible

Researchers have many tools to reduce the potential for false precision but three foundational techniques are particularly important.

First, use the largest sample sizes you can afford. While there is no “best” sample size that applies to every study, it’s fair to say more is better. In the market research space, 700 per group often offers the precision necessary to determine whether two groups are different - in this case, 10% vs 15% will probably be statistically different. When budgets lead to sample sizes of 200 to 300 per group, reliability will decrease and false precision will increase.

Second, use comparison or control groups as often as possible. Without a comparison, it’s impossible to know how much random chance affected the data. Was recall of your brand actually 10% or would 10% of people have recalled a brand you just made up? Did 10% of people try or buy or like or recommend your product or would 10% of people have said the same of a brand you just made up? No matter how careful they are, people will always misremember and misunderstand seemingly obvious things.

Third, when the opportunity arises, use a true random sample. If you’re lucky enough to be working with students registered at a school or cashiers employed in a store, it may be possible to gain consent from a sample of the population. Unfortunately, most market researchers won’t have access to a population list of customers/shoppers/buyers/users and so won’t be able to benefit from this.

Use as few significant digits as possible

Numbers are easy to generate. Throw a questionnaire at 700 people, run chi-squares, calculate pvalues, and build thousand-page tabulations. But those resulting numbers aren’t truth. They are representations of complex subjective constructs based on fallible, unreliable humans. Where truth is 68%, a survey result could be 61% or 69%. To say that 61.37% of people would recommend hypothetical Brand C is a gross misuse of decimal places.

Decimal places are perhaps the most problematic source of false precision, particularly in the marketing research world. To avoid this, don’t use any decimal places when percentage values are between 5% and 95%. Similarly, avoid using two decimal places when reporting Likert results. Only venture into one or more decimal places when you’re working with huge sample sizes from truly random samples.

Even better, if you’re brave and want to express your appreciation for false precision, round 61.37% to ‘about 60%.’

Use statistical testing wisely

Like artificial intelligence, statistical tests are meaningless when they aren’t offered with human oversight.

Tabulation reports can include thousands of t-tests and chi-square tests but, by design, we know that 5% of the significant results are Type I errors. Even worse, we don’t know which of those significant results are false. Because they are easy to find and exciting to report, it’s easy to overuse these significant results. To help readers grasp the concept of false precision, it’s a good idea to share corroborating trends from other sources such as last year’s research report, loyalty data, economic or political data.

If you’re lucky enough to be using random samples, always report margins of error. Further, always report available confidence intervals. While these numbers also incorporate a degree of false precision, readers need reminders that any statistics shared aren’t carved in stone.

Most importantly, ensure your reader understands that any numbers presented are not truth. Rather, they are vastly closer to truth than hypothesizing.

Summary

False precision is an easy trap to fall into, especially when the research results match your hypotheses. It can result in misleading interpretations, flawed decision-making, and ultimately, negative consequences for businesses. However, by being mindful of the limitations of research designs and data reporting, and offering clear instructions on how to best interpret numbers, researchers can help marketers better understand their data and make more informed and accurate decisions. If you’re curious about false precision might present itself in your research, feel free to connect with one of our survey experts!

Companies are in a heated race to showcase how their AI innovations help businesses achieve significant gains in processing data more quickly and accurately. But, as with all things, jumping on the bandwagon without a plan is rarely a wise move.

As AI tools developed in a variety of industries over the years, their use has uncovered lessons for future innovators. Here are just a few of those lessons.

Sudden pivots are wise

Though it seems like a lifetime ago, it was only eight years ago that Microsoft released the Tay chatbot. It instantly became a playful conversational tool with cute little quirks. However, people quickly migrated from having fun conversations with it to engaging in more controversial use cases. They realized they could train Tay, and soon taught it to become racist, sexist, and hateful.

This unanticipated outcome led to two important learnings. First, Microsoft reacted quickly and removed Tay from public use. Tay’s removal did not necessarily reflect a failure but, rather, a calculated risk within the innovation funnel. Second, as we’ve already learned from “Privacy by design” methods, the Tay incident reinforced the need for AI tools to incorporate “Ethics by design” models. Thanks in part to Tay, most AI tools now incorporate ethical guardrails. Take the risks, bring your innovations to market, but ensure they are prebuilt with processes to detect and prevent misuse.

Minimum viable standards are relative

Remember when restroom hand dryers with sensors first came out? They worked great for many people, but it soon became apparent that they were unreliable for people with darker skin tones. Developers hadn’t tested the product on people who had darker skin. Across the board, we’ve seen that AI tools are often biased towards pale, male faces because other people are excluded in sufficient quantities from AI training data. As a result, we now have higher minimum standards for training datasets, and we ensure they include people reflecting a wide range of demographics, especially in social and market research.

Our standards have improved over time, but they also differ based on the use case. In the research industry, for example, if you need to code questionnaire verbatims to understand which color of bar soap people prefer, 85% accuracy is suitable for the job. Increasing the 85% to 95% won’t change the outcome of the research but it will take longer and cost more. On the other hand, if you need to understand the efficacy of different mental health therapies, achieving 99% accuracy via automated coding enhanced with manual coding is the better way to go. Life-and- death situations necessitate higher accuracy. Standards are relative.

Ensure people retain final oversight

If you ask several competitive AI image generation tools to create an image of fish in a river, and they all show sushi and maki rolls swimming upstream, that doesn’t make the image valid. In fact, after seeing just one image, people would know the result was invalid.

This is exactly why people are necessary to confirm the validity and accuracy of AI tools. For example, during the development of our qualitative coding tool, Ascribe, we compared the results generated by the AI tool with results generated by expert human coders. It takes time to continually generate results in various industries and topics, and then test those results with human coding. But, that ongoing process is time well-spent to ensure that the quality of AI results is comparable to or better than human results.

Cautious risk-taking will win the AI race

Perfection is elusive, even in the age of AI. Every product, no matter how advanced, has its limitations. While some flaws, like those seen with Tay, might demand drastic changes, most can be addressed with small tweaks or by discovering the optimal use cases. The secret to successful innovation is a balance of bold ideas, agile adaptation, and the courage to take small, calculated risks. If you’re curious to learn more about our approach to using AI wisely, please get in touch with one of our Ascribe or Artificial Intelligence experts.

Research reports come in a full range of formats. From short and precise to long and detailed, there’s a report for that. In this post, we’ll outline the pros and cons of three formats to help you choose the one that will best suit your business needs.

Data reports are cheap, fast, and easy

Historically, the research industry has done a great job of producing reports with reams of data. Today, these types of reports are supported by automated systems that produce a nicely colored chart for every single question in a questionnaire. Supplemental charts might also be built manually using data pulled from tabulations. With charts in place, titles or headlines are easily prepared by converting the largest or smallest number in each chart to a phrase.

You know you’re working with a data report when the headlines read like these:

• 75% of shoppers seek out discounts
• 55% of people prefer package B
• 80% of people indicated child-care is a key issue

Data reports are fast and relatively cheap to create. Launch a survey, request the automated charts, and choose an interesting number to become the title on each slide. These reports require little thinking and creativity and, as such, nearly anyone can prepare them regardless of their skill or experience.

Despite their tendency to be quite long, these types of reports answer limited questions. And though automation means that they’re far less expensive, they are the least helpful and are rarely used again. To improve data visualization and streamline reporting, consider integrating dashboards using modern admin panel themes, a quick solution that provides interactive and user-friendly insights into your findings.

Insight reports take time, money, and care

In recent years, our industry has realized that data reports offer little value to research buyers. “What” without the “how, why, and when” doesn’t lead to repeatable business decisions that can be generalized for long-term value across products and audiences. Consequently, many report providers have turned their efforts from data reports to insight reports.

Insight reports require researchers to simultaneously consider multiple pieces of data and derive an understanding that isn’t inherently obvious in the data. It requires understanding the external, environmental context in which the research took place, and how this connects to the business problem at hand.

Insight reports are more expensive and take more time to create because they require more skill and experience to create. Rather than focusing on the largest or smallest number in a chart, researchers instead seek out the unexpected, nonsensical, out of place numbers which are most important. Turning those numbers into insight requires digging into external data – cultural, historical, geographical, political, technological, economic.

Headlines in an insights might look like this:

• Because of the sudden cost of living increase, 50% more shoppers are value seekers
• Despite being less accessible, people preferred Format B because their favorite celebrity was recently seen using it
• The need for child-care declined to 80% due to higher unemployment rates

Although insight reports take more time and money to create, they create value beyond the cost of data collection. They offer insights and understanding that can be used not only for the business problems at hand, but also for other challenges still to come with similar products, categories, or audiences. In the end, these more valuable insights lead to long-term loyalty between the researcher buyer and provider.

Action reports are expensive but they generate ROI

The most valuable reports are action reports. These reports go beyond data points and contextual insights to offer actionable recommendations that identify long-term solutions for specific and generalized business problems. If you were to map out the questions and slides in these types of reports, you wouldn’t find a chart for every question and the slides wouldn’t be ‘in order.’ Rather, you would find that each slide title contributes to an overall coherent story complete with an introduction, key issues, and final conclusions.

Here are some examples of action statements in a report

• With the increased cost of living creating more value seekers, switching to lower quality paper and using less ink on each printing will help decrease prices by 8%
• Despite lower ratings for Format A, it should be adopted with a marketing plan that demonstrates these 3 accessibility use cases
• Though demand for child-care has decreased, we must increase child-care openings to support job-seeking opportunities

Unlike lengthy and less expensive data reports, action reports are usually short and much more expensive mainly because they are vastly more difficult to create. Action reports depend on a true partnership between the research buyer and provider, a partnership wherein the provider has been privy to the confidential workings and challenges experienced by the research buyer. A true partnership leads to reports that offer long-term value and increasing ROI.

Key Takeaways

The choice between data reports, insight reports, and action reports ultimately boils down to your specific business needs. While data reports offer a quick and cost-effective solution, their limited insights may provide no actionable recommendations. Insight reports, on the other hand, dive deeper into the data, uncovering valuable patterns and trends. And, at their best, action reports truly deliver by providing not only insights but also concrete recommendations to drive business growth. The key to success lies in choosing the type of report that aligns with your strategic goals and provides the actionable insights you need to make informed decisions. If you’d like to understand more about research processes, connect with one of our survey experts. We always love chatting about the best ways to uncover valid and reliable insights!

Market researchers and data analysts are constantly looking for faster and more efficient ways to analyze and extract actionable insights from vast amounts of open-end feedback. The launch of Theme Extractor 2.0 and Ask Ascribe marks a major leap forward in open-end analysis, equipping organizations with advanced tools that offer exceptional speed, precision, and depth of insight.

Let’s explore how these innovative solutions enhance Coder, Ascribe’s verbatim coding platform, and CX Inspector, Ascribe’s text analytics solution, along with how they can improve your workflow.

What is Theme Extractor 2.0?

Theme Extractor 2.0 is Ascribe’s latest AI-powered innovation, designed to automatically analyze open-ended comments and survey responses with an accuracy of over 95%. It generates a human-like, richly descriptive codebook with well-structured nets, allowing users to spend less time on the technical aspects of analyzing open ends and more on interpreting results. By using Artificial Intelligence (AI) and Natural Language Processing (NLP), the tool navigates the complexities of text analysis, dramatically reducing manual analysis and delivering results with incredible speed.

Check out the theme-based codes and nets in the following snapshot of results, delivering a significant improvement over the one-word topics used in older technologies. A total of 1655 responses were analyzed, and 1638, or 99%, were immediately classified into 33 codes and 9 nets.

Benefits of Theme Extractor 2.0

1. Accelerated Workflow and Efficiency

Analyzing open-ended responses manually can be extremely time-consuming. Theme Extractor 2.0 automates this task, allowing users to process large datasets and free up valuable time for more in-depth analysis. With minimal manual intervention required, it streamlines your workflow and reduces the risk of errors.

2. Exceptional Accuracy

With over 95% accuracy, Theme Extractor 2.0 provides a level of precision that rivals human coding. By minimizing uncoded responses and overlapping codes, it ensures that your data is clean, structured, and primed for thorough analysis. You can trust that the data and insights are as accurate and reliable as possible.

3. Clear and Organized Codebook Structure

Theme Extractor 2.0 generates theme-based codes and nets that help uncover deeper patterns within your open-end responses. Say goodbye to one-word topics, and hello to helpful, descriptive results. It structures codes into clear, logical nets, making it easier for researchers to interpret data and spot key themes. These improved results add clarity to your analysis and ensure a seamless experience.

4. Minimal Manual Intervention

With minimal manual intervention needed, Theme Extractor 2.0 handles the labor-intensive tasks, allowing users to focus on more strategic work. Automating analysis allows researchers to quickly move from raw data to actionable results, saving time and reducing the risk of human error.

What is Ask Ascribe?

Ask Ascribe is an innovative AI-powered tool that lets you ask natural language questions about your data and get instant, actionable insights through answers, reports, and summaries. Powered by advanced Generative AI models, Ask Ascribe allows researchers to engage directly with their data, enabling them to ask specific questions and gain a deeper understanding of key themes, customer sentiments, and areas for improvement.

What is Ask Ascribe?

1. Instant Answers with Natural Language Interaction

You don’t have to sift through piles of data anymore—Ask Ascribe lets you ask natural language questions like “What are the main themes in this feedback?” or “How can I improve my Net Promoter Score (NPS)?” and receive accurate, data-driven answers in seconds. This AIpowered feature simplifies the analysis process, providing you with real-time insights into the story behind your data.

2. Uncover Actionable Insights Easily

Ask Ascribe goes beyond providing answers; it delivers actionable insights that empower you to make informed decisions. Whether you’re looking to explore customer emotions, identify pain points, or uncover opportunities for improvement, Ask Ascribe helps you quickly grasp what your data is revealing and what steps to take next.

3. Explore Data on a Deeper Level

With Ask Ascribe, you can easily drill down into responses to see the original comments tied to specific insights. This feature allows you to explore your data more thoroughly and gain a nuanced understanding of key drivers, customer feedback, and other qualitative insights. You’re not just scratching the surface—you’re engaging deeply with your data.

4. Empowers Better Decision-Making

By delivering quick, relevant answers to your questions, Ask Ascribe helps you make informed decisions based on real data. The ability to “interview” your data in real-time empowers researchers, analysts, and business leaders to respond faster and more strategically to emerging trends and customer feedback.

Why Theme Extractor 2.0 and Ask Ascribe Matter

Together, Theme Extractor 2.0 and Ask Ascribe – available as features in Ascribe’s Coder verbatim coding platform, CX Inspector text analytics and Services solutions – are changing how researchers and analysts engage with their data. By automating open-end analysis and coding processes and enabling AI-powered Q&A, these tools drastically reduce the time and effort needed to turn open-ended responses into clear results and actionable insights. Here’s why they make a difference:

• Unparalleled Speed to Results: Load up a data file or ask a question and the results appear within seconds. Now, any dataset with open-end responses of any size and complexity can be analyzed quickly and easily.
• Increased Productivity: These solutions streamline workflows by eliminating manual processes, enabling researchers to analyze more data in less time.
• Improved Accuracy: Both tools leverage advanced AI models to deliver precise, more descriptive results, ensuring you can trust the insights derived from your data.
• Deeper Understanding of Data: By offering theme-based codes and nets, and natural language interaction, researchers can dig deeper into their data, leading to more strategic decision-making.

Conclusion

The ability to transform open-ended data into clear, actionable results is essential for any organization that values customer feedback and market research. Ascribe’s Coder, CX Inspector, and Services, available with Theme Extractor 2.0 and Ask Ascribe, are powerful solutions that deliver a new level of speed, accuracy, and ease to researchers and analysts.

These solutions are the future of open-end data analysis—enabling users to unlock valuable insights with minimal effort and maximum efficiency. If you’re ready to transform how you work with data, it’s time to explore what Ascribe can do for you.

Interested in discussing how Ascribe can help you? Click here to sign up for a live demo with your data, or drop us your contact info and we will reach out to you.

Click here

Whether your environment is academic or industry, every social, market, and consumer insights expert has developed a unique set of skills shaped by their education and experience. Regardless of the depth or source of these skills, they remain, however, incomplete. With that in mind, here are some key skills that industry and academic researchers can learn from one another.

Embrace the randomness of real life

Every researcher loves a fully factorial, experimentally designed retail aisle, but those don’t exist in real life. Real-world shopping means not finding a product, brand, size, or format you’re looking for, coming upon unexpectedly low or high prices, and dealing with rude customers and employees.

Practical researchers who conduct on-site shop-alongs and ethnographies have extensive experience analyzing and interpreting complex, messy, real- life scenarios. Their hands-on experience makes their work highly relevant to business leaders needing to understand current industry needs, market trends, and consumer preferences. Real life may be messy, but academic researchers should learn to embrace a bit more mess.

Incorporate more theoretical depth

Human behavior isn’t new. For more than a hundred years, academic researchers have worked to understand market and consumer behaviors and to build theoretical foundations like Cognitive Dissonance Theory and the Diffusion of Innovation Theory that can be used to hypothesize and predict future behaviors. They’ve built on the work that came before them, knowing that it is the foundation of their own research. This is what elevates their work from simpler descriptive analyses and hypotheses to deeper understandings of who, what, when, where, why, and how certain consumer behaviors occur.

Rather than trying to understand a single problem, such as which package will be successful today, academic researchers work to build theories that will have a broader impact. By digging into the human behavior archives more often, industry researchers could also generate more robust conclusions and recommendations.

Practice agile research processes

Where academic researchers often have months or years to run a research project that accommodates a wide range of variables, industry researchers often have days or weeks. Industry researchers have learned to expect and adapt to changing circumstances so that they can meet rapid turnaround times. Their work is efficient and responsive to the real world, which can change literally overnight. Industry research is often simple and quickly actionable. Bang for the buck is clear and personally observable. Academic researchers could definitely benefit from tightening their timelines and getting their outcomes into the real world more quickly.

Engage More Stakeholders

Determining whether customers like package A or package B more is not as simple as it seems. Yes, a highly controlled experiment will reveal a winner, but simply moving forward with the results of what is essentially a customer vote could lead to a massive failure.

To ensure an experiment doesn’t land in the proverbial file drawer, industry researchers are careful to engage many of their stakeholders, including not only customers but also the package designer who will need to make any subsequent tweaks to their beloved design, the brand manager who campaigned for the losing package, the category manager who must budget for a more expensive package, the business development team who must promote a design they don’t personally like, and the executive leadership who is divided on the decision.

By engaging a cross-disciplinary team from the beginning, industry researchers have learned how to strengthen the applicability and reach of their research. Sometimes, academic researchers need to remember that uncovering truth isn’t the automatic path to success.

Take risks with new innovations

If you’re not already using AI, your competitors will pull ahead and leave you behind. That doesn’t mean you should jump onto the AI bandwagon and use it everywhere you possibly can. It does mean you should find ways to incorporate new methodologies like AI as soon as you find practical and valid uses for them.

Rather than waiting for academic researchers to complete highly controlled studies, industry researchers are incorporating AI tools and techniques for experimentation along the way. With side-by-side comparisons, industry researchers allow their customers to see and get comfortable with the results while also ensuring the innovations are valid and reliable in real life. The key is to take safe and considered risks along the way.

Value long-term learning

You’re a researcher because throughout your life, as a baby, toddler, child, teenager, and young adult, you were exposed to a set of experiences and circumstances that consciously or unconsciously led you to reject some job opportunities and choose this one. You are a lifelong experiment, not a one- time study from last week.

Similarly, academic researchers value lifelong experiences, which means that many of them conduct studies that take 5, 10, or even 50 years to complete. These studies help us understand systemic, longitudinal issues that are not visible in one-time, cross-sectional studies. Longitudinal studies are how we understand the impacts of early education on later voting patterns or early community service experiences on later-life consumer issues. Industry researchers would be well-served to take a longer-term approach to some of their studies.

Incorporate more scientific rigor

Academic researchers are rigorously trained in statistics, sampling, and research methodologies. Through years of schooling, they’ve learned to scrutinize and interpret data with a critical eye. They know when to use a Scheffé correction and why they won’t use it this time. They know the pitfalls of data tabulations and when to use Python and R scripts. Consequently, they succeed with high research quality and validity, minimal biases, and maximum reliability.

Given that many market and consumer researchers are serendipitous members of the industry and only receive on-the-job or as-needed training, this is definitely a gap that needs to be filled.

Summary

Whether you’re an academic or industry researcher, we all have knowledge gaps across a range of areas. Having a growth mindset focused on uncovering and filling those gaps is how good researchers eventually become great researchers. We would all benefit from taking a masterclass from ESOMAR or a course with the MRII, so don’t be shy. Have a look at their offerings and see how you can add new skills to your repertoire. Or, if you’d like to chat with a passionate colleague about research techniques, please connect with one of our survey experts. We’d love to hear from you!

Introduction

As the political landscape shifts beneath our feet, polling remains a critical tool for understanding public sentiment. What if the path to more accurate voter insights lies in learning from our past experiences?

To deepen our understanding, we had a conversation with Dr. Don Levy, Director of the Siena College Research Institute, who shared valuable perspectives from his extensive experience in polling. His insights, derived from our discussion as well as his appearances on podcasts with AAPOR and WXXI News, shed light on the factors that can skew polling results.

In this blog, we’ll explore key lessons from past elections and the innovative strategies pollsters like Dr. Levy are implementing to enhance their methodologies and restore public trust.

Key Lessons from the 2016 and 2020 Elections

The polling industry has learned vital lessons that have reshaped its approach to electoral forecasting. Dr. Don Levy discusses the factors contributing to bias observed in the last two elections:

“I think we all learned a great deal from 2016, which, at this point, feels like the distant past. There was an insufficient amount of state polling during that election, and some battleground states—like Michigan, Wisconsin, and, to some degree, Pennsylvania—were not even identified as such. Additionally, some polls did not weight responses by education in that cycle. We learned from those mistakes.

By 2020, the polling landscape had changed entirely. Everyone was considering education, recognizing it as one of the key fissures in the American electorate, and there was a greater focus on several key battleground states. We were not alone in polling Wisconsin, Michigan, and Pennsylvania at that point.”

Dr. Levy identifies two critical areas for improvement:

1. State-Level Polling:
   Many national polls failed to accurately gauge support for candidates due to an insufficient focus on state-level data. This oversight revealed how local dynamics and voter sentiment vary significantly across different regions, leading to inaccuracies in predictions.
2. Education as a Variable:
   The impact of education on voter preferences was underestimated in many polls. Dr. Levy notes, "Education proved to be a dividing line, especially in battleground states." This insight emphasizes the importance of moving beyond traditional demographics. Pollsters must now consider how educational background influences voter behavior to enhance the accuracy of their models.

‍

In response to these lessons, methodologies have evolved to prioritize regional polling and implement nuanced weighting practices that better represent educational backgrounds. By learning from past missteps, pollsters are striving to produce more reliable results, thereby reinforcing the credibility of polling as a tool for democratic engagement.

This recognition of education as a crucial variable, alongside the emphasis on state-level insights, marks a significant shift in voter sampling approaches. The goal now is to capture a clearer picture of public sentiment, ensuring that polling remains a trusted resource in the ever-changing political landscape.

Addressing Challenges with Innovative Approaches

In response to recent election cycles, pollsters have made significant strides to tackle key challenges. Dr. Levy highlights how these adjustments contribute to better accuracy and engagement:

• Achieving Diverse Representation Through Stratified Sampling: Pollsters use stratified sampling to ensure diverse representation across demographics, selecting sample groups that mirror the broader population—not only by age, gender, and race but also by considering regional and socioeconomic nuances. This helps create a more accurate model of voter intentions.

“We faced a bias in 2020. Our view was that ardent Trump voters—not the shy ones—tended not to respond to polls. This issue was evident across all polling methods: phone, web-based, text-to-web, and IVR. In analyzing about 50 polls from reputable AAPOR members in the ten days before the election, only three correctly predicted outcomes or leaned Republican, while nearly all others were biased the other way.

In Pennsylvania, for instance, areas with the highest Trump support showed significant polling errors. In regions where Trump won by 70% to 30%, polls often showed him at around 60% to 40%. But systemically, what occurred was that as we filled our quotas, we would just simply get a greater participation among Biden voters who were, to stereotype, white men without a college education than from a representative sample of Trump voters.

So, do we face that threat again this election? Yes. To address it, we’re implementing rigorous quota management, stratified sampling, and actively reducing drop-offs, along with the benefit of a repeat election.”

• Minimizing Drop-Offs by Leveraging Historical Data and Voter Lists: Pollsters have also become more vigilant about minimizing drop-offs, aiming to retain as many responses as possible to enhance data robustness. "Drop-offs can skew results by unintentionally filtering out certain groups," says Dr. Levy. As a solution, historical data and detailed voter lists have become valuable tools, allowing pollsters to refine their models and develop weighting strategies that are more reflective of the electorate. This approach helps account for variances in response rates, especially among underrepresented groups.
• Increasing Voter Engagement using AI and Data Enhancement: With advancements in artificial intelligence and data enhancement, pollsters now have more sophisticated tools to reach and engage voters, even as response rates remain challenging. AI-driven insights can optimize contact strategies, enabling pollsters to identify when, where, and how to engage different voter groups effectively.

These innovations collectively improve polling accuracy, reinforce public trust, and ensure that polling remains a valuable tool in capturing voter sentiment.

Upholding Integrity: The Importance of Non-Partisan Polling

In commitment to Non-Partisan Polling, maintaining public trust is paramount, and organizations like the American Association for Public Opinion Research (AAPOR) are essential in promoting transparency and upholding high standards in the polling industry. Dr. Levy highlights that these organizations work to enhance public understanding of non-partisan polling, which serves as a critical check against potential biases that can arise in politically charged environments.

“My hope—and my request—to my friends in the polling community is to broadcast to U.S. citizens and voters, explaining who we are, what we do, and why it’s important. It would be a positive act of citizenship to participate in high-quality, non-partisan political polls. We're not trying to sell you anything, convince you of anything, and/or manipulate you in any manner, shape or form.”

Pollsters are committed to contributing positively to the democratic process by fostering greater transparency around polling methods, data sources, and any limitations that may impact findings. Through collaboration and innovation, the polling industry continues to adapt and improve, addressing new challenges with integrity. As the political and social landscape evolves, the industry’s focus on high standards remains steadfast, ensuring that polling serves as an informative, unbiased reflection of public sentiment and helps advance informed democratic engagement.

Conclusion

Reflecting on the lessons of recent elections, it’s evident that the future of polling depends on how well the industry adapts and innovates. By embracing better sampling techniques and new technology, pollsters are working to paint a truer picture of where voters stand and what they care about. The path forward is one of continuous learning and evolution, ensuring that polling remains a trusted tool for understanding public sentiment.

Open-ended questions and analysis provide a rich layer of insights that goes beyond the limitations of structured data, offering a clearer understanding of customer sentiment, emerging trends, and overall experience. Whether gathered from surveys, customer reviews, or social media, this feedback can be a goldmine for business insight—but analyzing it effectively can be a challenge. For those dealing with thousands of open-ended questions and responses, traditional methods such as reading through open ends, tallying up responses in Excel, or simply not having the capacity or the right tools to analyze the feedback becomes overwhelming. Additionally, using generic AI tools like ChatGPT often proves inadequate. Without a qualified solution to easily analyze open-ended responses, researchers risk generating incomplete or inaccurate results, which can lead to delays and inefficiencies in completing projects.

At Voxco, one of the reasons we are excited about integrating Ascribe into our online survey platform is because of how Ascribe’s Coder and CX Inspector simplify and streamline open-end analysis, especially for corporate and market researchers. These tools leverage the latest advancements in artificial intelligence, making it easier to unlock actionable insights from vast amounts of open-ended data. This exciting product release showcases our joint commitment to offering innovative solutions that help researchers drive more impactful decisions.

Why Open-Ended Questions and Analysis Matter

• Deeper Customer Insights: Open-end analysis delivers a more comprehensive view of customer behaviors, motivations, and sentiments, helping companies better understand their audiences
• Uncover Hidden Insights: Open-ended responses can reveal unexpected insights that structured data misses, offering an early window into new trends and opportunities.
• Capture Nuances: Unlike structured responses, open-ended feedback allows customers to express their feelings and experiences in their own words, providing valuable context that helps businesses understand emotional drivers and pain points.
• Identify Emerging Trends: By analyzing open-ended feedback, businesses can spot new patterns that may not have been considered in the initial survey design, allowing them to respond quickly.
• Understand the ‘Why’: While quantitative data shows what’s happening, open-end feedback reveals why customers behave the way they do, offering more actionable insights.

How Ascribe’s Coder and CX Inspector Can Help

Ascribe’s industry-leading tools enable the analysis of open-ended feedback at scale with greater accuracy and speed by seamlessly blending artificial intelligence with human expertise. Here’s how Ascribe Coder and CX Inspector, both recently upgraded with Theme Extractor 2.0 and Ask Ascribe, empower researchers:

Theme Extractor 2.0

Automatically analyzes over 95% of open-ended responses and generates accurate, human-like codebooks. The tool eliminates overlap between themes and delivers cleaner, faster results—ideal for streamlining the research process.

Ask Ascribe

Ask Ascribe allows users to query their data in real-time using natural language. This interactive approach enables the quick identification of key themes, emotions, and areas for improvement, allowing businesses to act on insights faster.

Ascribe Coder: Enhancing Coding Productivity

Ascribe Coder enhances productivity by converting unstructured text into structured data. Here’s how:

• AI-Driven Human Coding: Combines AI with human intelligence to accelerate the coding process while maintaining precision.
• Customizable Automation: Allows users to adjust the level of automation based on project needs, ensuring control over cost, timing, and accuracy.
• Data Integration: Integrates open-ended feedback with survey data, providing a 360-degree view of the customer experience.

CX Inspector: Advanced Text Analytics for Deeper Insights

CX Inspector offers a robust platform for extracting and visualizing themes, sentiment, and emerging trends from open-end responses. Key features include:

• Instant Theme Extraction: Automatically identifies clear, descriptive themes from open-ended responses.
• Sentiment Analysis: Detects and visualizes customer sentiment instantly, enabling businesses to prioritize issues based on emotional impact.
• Actionable Insights: Combines theme detection and sentiment analysis to deliver clear, actionable insights that can be shared easily via dashboards and reports.

Conclusion: Unlock the Full Potential of Open-Ended Questions and Analysis

At Voxco, joining forces with Ascribe is part of our mission to empower researchers with powerful tools for analyzing open-ended feedback. As Rick Kieser, Chief Strategy Officer at Voxco, explains:

“Ascribe has dedicated 25 years listening to customer feedback and analyzing open-ended feedback, partnering with the world’s top market research firms and industry-leading corporations. By closely listening to the needs of these pioneers and continuously evolving, Ascribe has delivered cutting-edge solutions that shape the future of text analytics. The launch of Theme Extractor 2.0 and Ask Ascribe represents the pinnacle of this expertise—a culmination of decades of innovation, hard-earned insights, and the processing of over 6 billion customer comments. We’re excited to bring these solutions to Voxco’s customers and continue pushing the boundaries of innovation in research.”

With Ascribe Coder and CX Inspector, researchers can efficiently categorize and act on open-end feedback, driving more informed decisions and enhancing the overall customer experience.

Quantitative research is qualitative research in disguise

Whether you’re a market, academic, or social researcher, most human behavior researchers have a preference and expertise for quantitative or qualitative data collection tools. We tend to have preferences between focus groups and questionnaires, individual interviews and eye-tracking, ethnography and biometrics. We have a well-developed hammer, and we know how to make it solve most of our research problems.

However, the human experience is 100% qualitative, and quantitative research is really qualitative research in disguise. Researchers ask people to provide answers using distinct answer boxes, not realizing that they’re asking participants to pre-code highly subjective interpretations of complex experiences into imperfectly operationalized answer options. Those pre-coded answers are neither more precise nor valid than answering open-ended verbatims which are subsequently coded by the researcher. Whether the participant codes them, or the researcher codes them, both are representations of qualitative personal experiences crammed into a box.

We like to differentiate quantitative research as being measurable, structured, and controlled when qualitative research is also very much measurable, structured, and controlled. We like to say qualitative research is rich and in-depth when quantitative research can also be rich and in-depth. When well conducted, both qualitative and quantitative research give people the opportunity to reveal their underlying emotions and motivations. Quantitative research sometimes offers scale and statistical power, but the rest can be quite similar.

What can research users and practitioners do? Most importantly, we need to recognize that neither method is more valid, useful, nor important. It’s irrational to prioritize results from one method over the other. Second, research practitioners and users should have more than a basic level of training in both qualitative and quantitative research. It’s never a flex to be fluent in one method and mostly ignorant of the other. This only limits a researcher's perspective, problem-solving capabilities, and robustness of their findings.

Probability Sampling: A More Rigorous Form of Nonprobability Sampling

When it comes to choosing between probability and nonprobability sampling of human beings, the reality is that just about all sampling is nonprobability sampling. There are very few cases in which every member of population is known and every randomly selected participant consents. Probability sampling exists further along the continuum of nonprobability sampling.

For example, every student registered at a school can be identified but forcing a random sample of that population to participate in a study is impossible. Similarly, even with birth, death, driving, and voting records, it’s impossible to have a perfect list of every citizen in a city and subsequently force a random sample of those people to participate in a study. People will always be accidentally excluded and many who are included will not consent to participate. Nearly every attempt to achieve probability sampling with people is in fact an example of more rigorous nonprobablity sampling.

Regardless, probability sampling isn’t inherently superior to non probabliity sampling. Errors of sampling, data analysis, and interpretation creep into both methods. All participants behave on a continuum of honesty, attention, and care.

What can research users and practitioners do? In the end, the best sample is that one that is best suited for the job. Nonprobability samples are ideal for exploratory research, pilot studies, case studies, niche populations, trending, product testing, and, of course, working within time and budget constraints. If you require more precise statistical extrapolation such as for political polling, policy evaluation, market entry analysis, or demand forecasting, methods that approach probability sampling are preferred.

Every extrovert is an introvert

We love to classify people as male or female, introverted or extroverted, or online or offline shoppers. Our research results are massive collections of artificial, human-made binaries. But the human experience, even the most discrete physical attribute, exists on a continuum.

Binary groupings have a purpose and can be extremely helpful but it’s important to remember that we arbitrarily create the cut-points that become those binary groupings. We choose those cut points out of convenience not because they’re ‘true.’

No matter how we classify people into personality, demographic, shopping, social, or other groups, those groupings are artificial and they exist on a continuum. A group of ‘introverts’ could be subdivided into introverts and extroverts. And that subgroup of introverts could again be subdivided into introverts and extroverts, rinse and repeat. Being classified as a premium or budget shopper differs by who you’re shopping with, the product category, the time of day, and whether you’re hungry. Being classified as rural or urban can depend on political, national, local, and other characteristics.

What can research users and practitioners do? Remember that data tabulations are arbitrary and changeable. They can be redesigned once, twice, and thrice after the preliminary data has been reviewed. Design your initial tables with twice as many groups as will be necessary even if the sample sizes will be too small. With tables in hand, then you can evaluate the results and decide how many groups make sense and whether those groups should be equally sized.

Conclusion

Most binaries are arbitrary. They are human defined, human applied, and can be recoded into innumerable meaningful groups. While they are essential for simplifying the complexities of our world, every binary representation gives researchers a fresh opportunity to pause and question other equally valid categorizations that might exist. Questioning binaries is an important technique for researchers and marketers who want to reveal and explain the true complexities of consumer behaviors and preferences, ultimately improving the accuracy and relevance of marketing insights.

I’ve just crossed the six-month mark at Voxco and it’s been a whirlwind of a journey! I am loving getting to know all the people on our team, how they help our customers with a huge range of needs and challenges, and the potential we have together.

When joining a company as their new CEO, one of the first things I like to do after connecting with my team is to meet our customers, listen to industry experts, and hear from a broad range of stakeholders. What’s important to them? What challenges do they face? What gets them excited about going to work every day?

In talking with people at industry shows like Quirks and AAPOR, I immediately saw that AI has been embraced as a transformative market research technology that warrants significant investment. Exploring the capabilities of generative AI for enterprise applications can further revolutionize market research, enabling businesses to uncover innovative insights and optimize their decision-making processes. People are genuinely committed to the technology. For instance:

• A vast majority of trade show exhibitors have taken AI-forward approaches. And, at least partially because this is what conference committees are seeking, presenters too are taking an AI-forward approach.
• Whether their key services include data quality, sampling, analytics, reporting, or something else, most research providers are actively running internal AI projects, often leveraging tools like AI Chat to enhance their processes. About half of those projects are purely experimental but the other half are already customer-facing and revenue- generating.

Showing and discussing applications of AI in market research, however, is just noise. We need to understand the type and magnitude of impact that AI technologies have. In order to avoid long-term harm, we need to proactively measure, understand, and work toward preventing the misuse of AI. This can happen in several different ways.

1. Poor data input: Generative AI has many strengths, but it can also lead to data quality issues. Just as a data analyst knows that poor sampling practices and small sample sizes create large error rates and minimal generalizability, the same is true for GenAI. “Hallucinations” destroy validity, generate incorrect insights, and lead to poor business decisions. AI researchers need to identify and prevent all types of substandard data practices that can mislead AI processes. Companies that outsource digital marketing should be particularly cautious, as the quality of AI-generated content can significantly impact their online presence and brand reputation.
2. Misplaced applications: Because AI is amazing in many circumstances, it’s easy to run with it rather than trusting our gut and years of experience. Sometimes, training data doesn’t include the core data needed for making correct inferences. Sometimes we use a generalist AI tool over a research-specific AI tool. Researchers need to address the strengths and weaknesses of any AI tool they use to ensure unconscious biases that lead to incorrect business decisions are avoided.
3. Lack of validation: Researchers love data, experimentation, and validation. However, AI is still developing, and there's limited market data to validate new techniques. We don’t yet know if an approach that worked for one ad test will be effective across categories, target audiences, regions, and objectives. This calls for extensive documentation and robust databases.

Of course, there are some immensely valuable and already validated uses for AI tools. Tools like Ascribe (newly acquired as part of the Voxco platform) have already helped the research industry solve a long- running problem of avoiding coding open-ends simply because of time and cost constraints. Given that many questionnaires have ten or more short open-ends plus several long open-ends, this used to be a disappointing waste of respondent time and a loss of valuable insights for brands. This is one big problem solved.

I look forward to seeing how AI continues to evolve to create better business operations, research processes, and exceptional customer experiences. With a proactive approach to quality and validation, the opportunities are endless. I’d love to learn about your AI experiences so please feel free to connect with me on LinkedIn or talk with one of our survey experts.

Qualitative data, especially open-ended responses, can provide deep insights into consumer behavior, preferences, emotions, and desires. However, extracting actionable insights from qualitative data is more difficult than briefly skimming through survey responses or counting frequency, instead, it requires quantitative analysis of the open-ended responses in your data set and then effective visualization of the results. 

The Power of Visualizations

Visualizations play a crucial role in transforming raw, unstructured data into meaningful insights to help drive business decisions. When it comes to open-ended responses, visualizing the data with infographics, charts, and other visual options helps with identifying patterns, trends, and sentiments that may not be immediately apparent from reading the text alone. 

Visual tools have the ability to take vast, complex amounts of qualitative data like consumer comments and transform it into easily digestible, visually appealing formats. By presenting data in this way, these tools enable you to quickly identify key themes and ideas that might otherwise be buried in lengthy text responses. This streamlined approach not only saves time but also enhances your ability to make informed, data-driven decisions with confidence, ensuring that critical insights from your respondents are not overlooked.

Understanding Your Open-End Data

Before creating your data visualizations, it’s essential to understand the nature of your open-end data. Open-ended responses often vary in length and complexity, ranging from short, single-word answers to detailed narratives. This variability makes it necessary to prepare your data carefully to ensure accurate and insightful visualizations.

Preparing Data for Visualization

Preparation is a critical first step in the data visualization process. This phase involves not only the collection of data but also a thorough review and organization of that data to ensure it’s ready for effective visualization.

Initially, it’s essential to gather all relevant data from your surveys, interviews, or other qualitative research sources. Once collected, the data needs to be carefully reviewed, which includes cleaning and organizing the responses. This might involve filtering out irrelevant, duplicate, or "spam" responses that could skew your results, ensuring that only accurate and meaningful data is included in your analysis.

Next, responses should be classified into themes, which is the process of categorizing similar pieces of information under unified topics or concepts. This helps to distill large amounts of qualitative data into more manageable and understandable segments. Additionally, tagging specific segments of responses that are particularly relevant to your analysis can help highlight key insights and trends that align with your research objectives.

It’s also important to consider the context of your data, especially in relation to the research questions your organization has posed. The framing of these questions can significantly influence the way responses are interpreted. By keeping these questions in mind, you can ensure that your visualizations will address the core issues you set out to explore.

Proper preparation lays the groundwork for creating visualizations that accurately reflect the underlying sentiments and themes within your data. This meticulous approach not only enhances the clarity and effectiveness of your visualizations but also provides a robust foundation for deeper data analysis, enabling you to draw more reliable and actionable insights from your quantitative and qualitative research questions.

CX Inspector: A Leading Data Visualization Tool

When it comes to visualizing open-end data results, CX Inspector stands out as a top tool for researchers and analysts. CX Inspector is designed to simplify the complex process of quantitative and qualitative data analysis. Offering advanced features like generative AI and theme extraction, combined with its intuitive interface and powerful analytical capabilities, CX Inspector allows you to import, analyze, and visualize the results with minimal effort, making it an indispensable tool for any research project with qualitative data like open end responses.

Theme and Sentiment Analysis for Qualitative Data Visualization

One of the most effective ways to visualize open-end data is through theme and sentiment analysis. These techniques help you identify the underlying patterns and emotions in your data, which can then be represented visually. Here are some popular methods:

Word Clouds

Word clouds are a simple yet powerful way to visualize the most common words or phrases in your open-end responses. They provide a quick overview of the key themes by displaying words in varying sizes based on their frequency. While word clouds are great for initial exploration, they may oversimplify the data, so it's important to use them in conjunction with more detailed analyses.

Thematic Clustering

Thematic clustering involves grouping similar responses into clusters based on shared themes. This method is particularly useful for identifying patterns and trends in large datasets. By visualizing these clusters, you can easily see which themes are most prominent and how they relate to one another.

Network Diagrams

Network diagrams are another advanced visualization technique that shows the connections between different themes or keywords in your data. These diagrams are especially useful for understanding the relationships and interdependencies between various concepts, providing a more nuanced view of your data.

Frequency Distribution Graphs

Frequency distribution graphs, such as bar charts or histograms, are ideal for visualizing the prevalence of specific themes or sentiments in your open-end data. These graphs provide a clear, quantitative representation of how often certain responses or themes occur, making it easier to compare and contrast different aspects of your data.

Best Practices for Creating Effective Visualizations

Creating effective visualizations requires more than just choosing the right tool or method. It also involves adhering to certain best practices to ensure your visualizations are clear, accurate, and actionable.

Choosing the Right Visualization Method

The first step in creating effective visualizations is selecting the appropriate method based on your research goals and the nature of your data. For example, use word clouds for a high-level overview, network diagrams for exploring relationships, and frequency graphs for detailed quantitative analysis.

Ensuring Clarity and Accuracy in Qualitative Research

Clarity and accuracy are paramount in qualitative research. Ensure that your visualizations accurately represent the data by avoiding common pitfalls like overgeneralization or misinterpretation. Always double-check your data preparation and coding processes to maintain the integrity of your insights.

Making Visualizations Actionable

Finally, your visualizations should be actionable. This means they should not only provide insights but also guide decision-making. Consider how your audience will use the research findings presented and tailor your visualizations to highlight the most critical findings.

Begin Visualizing Qualitative Data with CX Inspector

Ready to transform your open-end data into powerful visualizations? CX Inspector makes it easy to analyze, visualize, and extract meaningful insights from your surveys with qualitative data. Whether you're looking to create word clouds, thematic clusters, or frequency graphs, CX Inspector provides the tools and support you need to succeed.

Don’t let valuable insights remain hidden in your data. Start visualizing with CX Inspector today and unlock the full potential of your open-end responses.