A/B-тесты

$\hat{p}=\frac{X}{n}$

$\text{uplift}_{\%}=\frac{\hat p_B-\hat p_A}{\hat p_A}\cdot 100\%$

$\Delta = \hat p_B-\hat p_A$

$SE(\hat p)=\sqrt{\frac{\hat p(1-\hat p)}{n}}$

$z = \frac{\hat p_B-\hat p_A}{SE_{\Delta}}$

$p \approx 2\,(1-\Phi(|z|)),\; \text{а без калькулятора: }|z|\approx1{,}64\Rightarrow p\approx0{,}10,\;1{,}96\Rightarrow0{,}05,\;2{,}58\Rightarrow0{,}01$

$(\hat p_B-\hat p_A)\pm z_{1-\alpha/2}\cdot SE_{\Delta}$

$n \approx \frac{(z_{1-\alpha/2}+z_{1-\beta})^2\left[p_A(1-p_A)+p_B(1-p_B)\right]}{MDE^2},\; n_A=n_B=n$

$MDE = (z_{1-\alpha/2}+z_{1-\beta})\sqrt{\frac{\hat p_A(1-\hat p_A)}{n_A}+\frac{\hat p_B(1-\hat p_B)}{n_B}}$

$\text{Power} \approx 1-\Phi\left(z_{1-\alpha/2}-\frac{|\Delta|}{SE_{\Delta}}\right),\;\beta\approx\Phi\left(z_{1-\alpha/2}-\frac{|\Delta|}{SE_{\Delta}}\right)$