1.79%Control (PSA) conversion
2.55%Treatment (Ad) conversion
+43.1%Relative lift
0.60 to 0.9495% CI of difference (points)
< 0.001p-value (z = 7.37)
Result: the ads increased conversion
Two-proportion z-test and chi-square test both reject the null hypothesis at alpha = 0.05. The test needed about 5,628 users per group for 80% power; the smaller group had 23,524.
Conversion rate by group
Error bars show 95% confidence intervals.
Conversion rate by ad exposure
Ad group. Correlation only: heavy browsers see more ads.
Conversion rate by day of week
Ad group. Monday and Tuesday convert best.
Conversion rate by hour of day
Ad group, hour in which the user saw the most ads.