One option vs. multi-options model

from prayas import *

The one-option model is a special case of the multi-options model. This notebook demonstrates this on a simple example.

Models

We define a one option model with three variants:

m1 = OneOptionModel(["Red", "Green", "Blue"], baseline="Red")

m1.set_result(successes=[209, 330, 408],
              trials=[236113, 236108, 243241])

We define a multi-options model with the same setup but with only one option per variant:

m2 = MultiOptionsModel(["Red", "Green", "Blue"], [1, 1, 1], baseline="Red")

m2.set_result(successes=[[209], [330], [408]],
              trials=[236113, 236108, 243241])

Comparison

m1.score_baseline()

	Variant	Measure	ProbabilityToBeBest	ProbabilityToBeatBaseline	UpliftFromBaseline	PotentialLossFromBaseline	MaxUplift	MaxPotentialLoss
0	Blue	conversion	0.9924	1.0	89.057671	0.0	88.963086	0.017103
1	Green	conversion	0.0076	1.0	57.726761	0.0	57.596831	16.614105
2	Red	conversion	0.0000	0.0	0.000000	0.0	-36.546947	47.095717

m2.score_baseline()

	Variant	Measure	ProbabilityToBeBest	ProbabilityToBeatBaseline	UpliftFromBaseline	PotentialLossFromBaseline	MaxUplift	MaxPotentialLoss
0	Blue	conversion	0.99355	1.0	89.123055	0.0	89.117280	0.017340
1	Green	conversion	0.00645	1.0	57.717259	0.0	57.636056	16.714971
2	Red	conversion	0.00000	0.0	0.000000	0.0	-36.562736	47.132823

We can see that the results are basically the same.