Netpeak ProphetFor those who are fond of testing we made this free online calculator. You can compare several tested variants by probability to beat all others. As a result, all you need is to look at only one number without any complicated math knowledge to make right decision.
|#||Successes||Trials||Frequency||Chance to beat all|
Netpeak Prophet is a free online statistics calculator. Initially designed for internet marketers, this service may be effectively used by specialists in many other fields. By using it, you can compare several variations and assess the probability that one of them is better than the rest.
The main Netpeak Prophet advantage over similar services is the probability calculation showing which variation is the best of the bunch. Compared with confidence intervals, this service is 10 times less complicated and time consuming. Besides, it doesn’t require any special mathematical background. All you need to do is just have a look at one number instead of conducting a full-featured research.
Netpeak Prophet is able to calculate the "Chance to Beat All". Other programs are capable of comparing only 2 variations (Chance to Beat Baseline/Origin). Furthermore, all statistics calculators (programs where you can enter your data) can only calculate the confidence intervals, but not the chances.
Since there is no synchronizing with the server, all calculations are performed immediately. However, when running slow PCs with old browsers the calculation may take up to 10 seconds.
Enter the number of successes and the total number of experiments.
- For CTR - successes are clicks, and experiments are impressions.
- For conversion percentage - successes are conversions, and experiments are click-throughs.
- It is also possible to calculate the other variations of binomial distribution.
- Click on "Calculate".
- Look at the column "Chance to Beat All". There is a high probability that this variation is better than the rest. If the variation’s "Chance to Beat All" = 90%, it means that with 90% probability this one is better than the others.
- If the number of experiments and variations differs by several times - then go to options and turn on the alignment. If in both cases the variation demonstrates high chances than it could be the best existing variation.
To remove the variation, delete the number of experiments or successes. Consequently, a blank variation entry will not be calculated.
Chance to Beat All
The "Chance to Beat All" is calculated using the probability theory. This parameter is as accurate as comparison of confidence intervals. At the same time it is more unambiguous and does not depend upon the choice of the significance coefficient.
However, in some situations it may be calculated incorrectly:
- f the number of experiments exceeds several millions, it is necessary to increase the number of points in the settings. Otherwise, the total sum of chances may deviate from 100%.
If the number of experiments and variations differs by several times.
- In this case, comparing the confidence intervals is too risky.
- Solution: it is necessary to align the number of variations. For example, the first variation shows 201 clicks per 1,003 impressions, the second - 21 per 101. So, you need to divide the first variation by 10 in order to get 20 per 100. You can find the corresponding option in the settings.
- The fact that you have different variations with different number of experiments indicates the violation of the experiment’s rules. Make sure you have collected data for all variations at once.
You can just add up "Chance to Beat All" parameter from different rows. It will be equal to 100%. However, you should be aware that we usually round off the results to the nearest whole number and the total sum may slightly differ from 100%.
Most calculators determine the frequency confidence intervals and we determine the Bayes’ ones. Consequently, a slight difference may occur.
With the standard number of points the Graphs tab may become quite unshapely even with 100,000 performed experiments. This doesn’t mean that calculations are incorrect. Due to some properties of the graphs drawing library, we have to send 200 times less points there.
Graphs are needed to facilitate the understanding and not to make a quantitative assessment.