Visitors
Conversions
Control
Variation
Find the best survey software for you!
(Along with a checklist to compare platforms)
Take a peek at our powerful survey features to design surveys that scale discoveries.
Explore Voxco
Need to map Voxco’s features & offerings? We can help!
Find the best customer experience platform
Uncover customer pain points, analyze feedback and run successful CX programs with the best CX platform for your team.
We’ve been avid users of the Voxco platform now for over 20 years. It gives us the flexibility to routinely enhance our survey toolkit and provides our clients with a more robust dataset and story to tell their clients.
Steve Male
VP Innovation & Strategic Partnerships, The Logit Group
Explore Regional Offices
Find the best survey software for you!
(Along with a checklist to compare platforms)
Take a peek at our powerful survey features to design surveys that scale discoveries.
Explore Voxco
Need to map Voxco’s features & offerings? We can help!
Find the best customer experience platform
Uncover customer pain points, analyze feedback and run successful CX programs with the best CX platform for your team.
We’ve been avid users of the Voxco platform now for over 20 years. It gives us the flexibility to routinely enhance our survey toolkit and provides our clients with a more robust dataset and story to tell their clients.
Steve Male
VP Innovation & Strategic Partnerships, The Logit Group
Explore Regional Offices
Identifying promoters and detractors has never been so easy! Voxco’s best-in-class solutions help you gather feedback from customers continually to understand their pain points with your product or service. Get ready to deliver a personalized and unique customer experience for increased customer satisfaction and loyalty! All you need to do is find your Net Promoter Score (NPS) by using our free NPS Calculator. Start by filling in the number of times you received a particular score in the boxes given below.
Visitors
Conversions
Control
Variation
Conversion Rate
Standard Error
Control
Variation
Z-Score
P-Value
Significant at % Confidence
Conversion Rate Limits
From
To
Control
Variation
What is statistical significance?
Statistical significance in A/B testing deals with the difference between the control version and the test version in your experiment, and that it is not due to error or random fate.
This is ensured by using a significance level. A significance level of 98% means that one can be 98% confident that any difference between your control version and the test version is real.
Statistical significance comes in handy when a business wants to observe how a change in their product or service (the experiment) can affect their business. Is there a positive or negative brought on by that change, and if there is, then why?
Statistical significance is used to ensure that any data you collect falls well within the margin of error you have deemed acceptable (signified by a confidence level), and that any final data is not one prone to error.
The first stage in A/B testing (or finding out statistical significance) is to formulate a hypothesis. There is a null hypothesis (H0, 0 being ‘nought’) and an alternative hypothesis. Typically the null hypothesis states that there is no relationship between the variables that you are comparing. The alternative hypothesis tries to prove a relationship exists and that the “test” is successful.
In A/B testing, there can be many instances that would work as a hypothesis – like adding a button on a website or an app, changing the UI or layout or color scheme and testing if these changes are affecting conversion rates by showing some users the normal (control) version.
A z-score is used to test the veracity of your null-hypothesis.
A p-value signifies the strength you have in favor of your hypothesis.
In A/B testing you must also decide if you are going to conduct a one-tailed or two-tailed test. One tailed tests can only account for directional effects from your alternative hypothesis. Two tailed tests on the other hand, also account for the eventuality that your hypothesis may have a negative impact. It is the safer approach.
Although A/B testing is an excellent technique for trying out changes and updates to your product, you need to go about conducting it the right way. There are a few techniques and guidelines you can keep in mind while conducting A/B tests.
Increasing sample size
The more people that partake in your tests, the more accurate the insights you will receive. For A/B testing, it means that you run your tests for a longer period of time, giving more people the chance to test out your null and alternative hypothesis.
Artificially direct traffic
You can add more links to your test pages on your social media and on your website to direct more traffic to them, allowing for a stress test of sorts.
Try significant changes
You can try making bigger changes to your product and testing its’ shock value on your users. This doesn’t mean a simple change to your color palette. Try getting users to engage with your services in an entirely new manner.
Don’t assume anything
Even if one scenario performs better than the other, it may not signify that users actually prefer using it. Therefore you must also pair A/B testing along with online surveys to get a deeper understanding into your users, and whether your A/B testing has yielded actionable insights.
Variation
We use cookies in our website to give you the best browsing experience and to tailor advertising. By continuing to use our website, you give us consent to the use of cookies. Read More
Name | Domain | Purpose | Expiry | Type |
---|---|---|---|---|
hubspotutk | www.voxco.com | HubSpot functional cookie. | 1 year | HTTP |
lhc_dir_locale | amplifyreach.com | --- | 52 years | --- |
lhc_dirclass | amplifyreach.com | --- | 52 years | --- |
Name | Domain | Purpose | Expiry | Type |
---|---|---|---|---|
_fbp | www.voxco.com | Facebook Pixel advertising first-party cookie | 3 months | HTTP |
__hstc | www.voxco.com | Hubspot marketing platform cookie. | 1 year | HTTP |
__hssrc | www.voxco.com | Hubspot marketing platform cookie. | 52 years | HTTP |
__hssc | www.voxco.com | Hubspot marketing platform cookie. | Session | HTTP |
Name | Domain | Purpose | Expiry | Type |
---|---|---|---|---|
_gid | www.voxco.com | Google Universal Analytics short-time unique user tracking identifier. | 1 days | HTTP |
MUID | bing.com | Microsoft User Identifier tracking cookie used by Bing Ads. | 1 year | HTTP |
MR | bat.bing.com | Microsoft User Identifier tracking cookie used by Bing Ads. | 7 days | HTTP |
IDE | doubleclick.net | Google advertising cookie used for user tracking and ad targeting purposes. | 2 years | HTTP |
_vwo_uuid_v2 | www.voxco.com | Generic Visual Website Optimizer (VWO) user tracking cookie. | 1 year | HTTP |
_vis_opt_s | www.voxco.com | Generic Visual Website Optimizer (VWO) user tracking cookie that detects if the user is new or returning to a particular campaign. | 3 months | HTTP |
_vis_opt_test_cookie | www.voxco.com | A session (temporary) cookie used by Generic Visual Website Optimizer (VWO) to detect if the cookies are enabled on the browser of the user or not. | 52 years | HTTP |
_ga | www.voxco.com | Google Universal Analytics long-time unique user tracking identifier. | 2 years | HTTP |
_uetsid | www.voxco.com | Microsoft Bing Ads Universal Event Tracking (UET) tracking cookie. | 1 days | HTTP |
vuid | vimeo.com | Vimeo tracking cookie | 2 years | HTTP |
Name | Domain | Purpose | Expiry | Type |
---|---|---|---|---|
__cf_bm | hubspot.com | Generic CloudFlare functional cookie. | Session | HTTP |
Name | Domain | Purpose | Expiry | Type |
---|---|---|---|---|
_gcl_au | www.voxco.com | --- | 3 months | --- |
_gat_gtag_UA_3262734_1 | www.voxco.com | --- | Session | --- |
_clck | www.voxco.com | --- | 1 year | --- |
_ga_HNFQQ528PZ | www.voxco.com | --- | 2 years | --- |
_clsk | www.voxco.com | --- | 1 days | --- |
visitor_id18452 | pardot.com | --- | 10 years | --- |
visitor_id18452-hash | pardot.com | --- | 10 years | --- |
lpv18452 | pi.pardot.com | --- | Session | --- |
lhc_per | www.voxco.com | --- | 6 months | --- |
_uetvid | www.voxco.com | --- | 1 year | --- |