Module model_conversion¶
-
class
prayas.model_conversion.ConversionModel¶ Base class for conversion-based models
This class defines the basic properties each conversion-based model needs to define as well as methods that either are common to all conversion-based models or need to be implemented specifically.
-
property
baseline¶ Returns or sets the baseline variant
-
property
primary_measure¶ Returns or sets the primary measure; default is conversion
-
property
loss_threshold¶ Returns or sets the loss threshold; default is 0.05
-
abstract
add_measure(name, success_value, nonsuccess_value=None)¶ Method to add an additional measure
- Parameters
name – Name of the measure, e.g., revenue
success_value – Value in case of success for each variant and option
nonsuccess_value – Value in case of nonsuccess for each variant and option
- Returns
None, used for its side-effect
-
abstract
sample(measure=None, n=20000)¶ Sample from the posterior distribution of each variant
- Parameters
measure – Name of the measure; default is conversion
n – Number of samples to draw
- Returns
Array with samples for each variant
-
abstract
measure(measure=None)¶ Return the value of a specific measure
- Parameters
measure – Name of the measure; default is conversion
- Returns
Data frame with measure per variant
-
abstract
plot(n=20000)¶ Plot the posterior distribution of each variant
- Parameters
n – Number of samples to draw
- Returns
Matplotlib pyplot object
-
abstract
set_result(successes, trials)¶ Set the result of the experiment
- Parameters
successes – Successes per variant and option
trials – Trials per variant and option
- Returns
None, used for its side-effect
-
abstract
new(successes, trials)¶ Create a new ConversionModel object, used for experiments
- Parameters
successes – Successes per variant and option
trials – Trials per variant and option
- Returns
ConversionModel object
-
score_pairwise(n=20000, drop=True)¶ Compute pairwise comparisons between all variants
- Parameters
n – Number of samples from posteriors as basis for comparison
drop – If True show only entries with Uplift > 0
- Returns
Data frame with
Left: First variant of the pairwise comparison
Right: Second variant
Measure: Comparison measure
LeftMeasure: Value of left
RightMeasure: Value of right
Uplift: Uplift of left in percentage
Loss: Potential loss in percentage if we go with left
ProbabilityUplift: Probability of the uplift
ProbabilityLoss: Probability of the loss if we go with left
LeftMeasureMaxLoss: Maximum loss if we go with left (subtracted from LeftMeasure)
Score: Score based on uplift * probability of the uplift
-
score_baseline(n=20000)¶ Compute baseline comparisons for all variants
- Parameters
n – Number of samples from posteriors as basis for comparison
- Returns
Data frame with
Variant: Name of the variant
Measure: Comparison measure
ProbabilityToBeBest: Probability to be the best among all variants
ProbabilityToBeatBaseline: Probability to be better than the baseline
UpliftFromBaseline: Uplift from baseline in percentage
PotentialLossFromBaseline: Potential loss if we go with the variant in percentage
MaxUplift: Maximum uplift in percentage
MaxPotentialLoss: Maximum loss in percentage
-
decision(n=20000, ignore_primary=False, ignore_loss_threshold=None)¶ Filter the result from the score_baseline method based on the defined experiment setup, i.e., the primary measure and the loss threshold
- Parameters
n – Number of samples from posteriors as basis for comparison
ignore_primary – If True ignores the primary measure filter
ignore_loss_threshold – If True ignores the loss threshold
- Returns
Data frame as returned by score_baseline with filtered rows
-
property