Let's create the variants
Posted: Sat Dec 28, 2024 10:42 am
How to Analyze A/B Test Results
When analyzing A/B test results, we need to focus on understanding them, not just whether the test was successful or not.
If we build our hypothesis correctly, even the losing variant will be a positive result , allowing us to gain insights that can be used in future testing and other areas of our business. We should not ignore negative results and losing variants.
Sometimes a test fails overall, but succeeds with at least one segment of visitors. Analysis is not simply about whether the A/B test was successful or not, but something much deeper. Let's focus on the information and segment the data , to uncover more information hidden beneath the surface.Let's also remember to set the numerical objective based on the history : before launching the A/B test, it is always a good idea to consult the previous results.
Once the hypothesis has been established, we must focus on the distinctive elements of the two variants , which can be: the image, the title or the text written in a paragraph rather than their formatting, the button of a form, the arrangement of an element on a page, but also two web pages in the desired versions, etc.
The original A and the variant B, whether pages or ads, will have to be created and published but above all managed in a precise and punctual way, to collect all the useful data deriving from the experiment.
Some tools help in this phase: Google Optimizer, VWO Virtual Website germany telegram data Optimizator or the services offered by advertising platforms, such as Google AdWords or Facebook Ads.
If we test one element at a time (position of a module), or a combination of elements that are closely connected to each other (position and button of a module), we will be able to draw more precise conclusions.
We evaluate the sample of recipients
In general, the principle applies that the larger the sample size , the more reliable the results we obtain will be.
There is no exact number, it depends a lot on the improvement we want to achieve and the level of significance desired. There are several calculators online (e.g. Unbounce, Evan Miller and others) that in a few seconds are able to determine the right sample size and the minimum time to run the test.
The data we need to be able to use one of these calculators are the following:
When analyzing A/B test results, we need to focus on understanding them, not just whether the test was successful or not.
If we build our hypothesis correctly, even the losing variant will be a positive result , allowing us to gain insights that can be used in future testing and other areas of our business. We should not ignore negative results and losing variants.
Sometimes a test fails overall, but succeeds with at least one segment of visitors. Analysis is not simply about whether the A/B test was successful or not, but something much deeper. Let's focus on the information and segment the data , to uncover more information hidden beneath the surface.Let's also remember to set the numerical objective based on the history : before launching the A/B test, it is always a good idea to consult the previous results.
Once the hypothesis has been established, we must focus on the distinctive elements of the two variants , which can be: the image, the title or the text written in a paragraph rather than their formatting, the button of a form, the arrangement of an element on a page, but also two web pages in the desired versions, etc.
The original A and the variant B, whether pages or ads, will have to be created and published but above all managed in a precise and punctual way, to collect all the useful data deriving from the experiment.
Some tools help in this phase: Google Optimizer, VWO Virtual Website germany telegram data Optimizator or the services offered by advertising platforms, such as Google AdWords or Facebook Ads.
If we test one element at a time (position of a module), or a combination of elements that are closely connected to each other (position and button of a module), we will be able to draw more precise conclusions.
We evaluate the sample of recipients
In general, the principle applies that the larger the sample size , the more reliable the results we obtain will be.
There is no exact number, it depends a lot on the improvement we want to achieve and the level of significance desired. There are several calculators online (e.g. Unbounce, Evan Miller and others) that in a few seconds are able to determine the right sample size and the minimum time to run the test.
The data we need to be able to use one of these calculators are the following: