Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update beta distribution parameters #643

Merged
merged 3 commits into from
Oct 4, 2024

Conversation

MustaphaU
Copy link
Contributor

@MustaphaU MustaphaU commented Oct 1, 2024

Issue #, if available:

Description of changes:

Update the beta distribution parameters in the simulate_experiment function to avoid bias towards lower success probability.

The current specification of the beta distribution:

theta = np.random.beta(conversions + 1, exposures + 1)

treats every exposure as a failure, that is overstates the failures thus undervalues the success probabilities of the variations. The effect is pronounced for variations with very high baseline conversion rates but less severe for variations with extremely low conversion rates.

Traditionally, the Thompson Sampling Algorithm for the Bernoulli Bandit Thompson Sampling algorithm is:

$$\begin{align*} 1: & \text{for } t = 1, 2, \ldots \text{ do:} \\\ 2: & \quad \quad \text{Sample model:} \\\ 3: & \quad \quad \text{for } k = 1 \text{ to } K \text{ do:} \\\ 4: & \quad \quad \quad \text{Sample } \theta_k \sim \text{beta}(\alpha_k, \beta_k) \\\ 5: & \quad \quad \text{end for} \\\ 6: & \quad \quad \text{Select and apply action:} \\\ 7: & \quad \quad x_t \leftarrow \arg\max_k \theta_k \\\ 8: & \quad \quad \text{Apply } x_t \text{ and observe } r_t \\\ 9: & \quad \quad \text{Update distribution:} \\\ 10: & \quad \quad (\alpha_{x_t}, \beta_{x_t}) \leftarrow (\alpha_{x_t} + r_t, \beta_{x_t} + 1 - r_t) \\\ 11: & \text{end for} \end{align*}$$

Where α, β represent the parameters of each arm i.e. the success and failure counts, respectively OR the number of conversions and non-conversions, respectively.

non-conversions (or beta)  = exposures - conversions

Description of testing performed to validate your changes (required if pull request includes CloudFormation or source code changes):

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

…iment.ipynb

Update the beta distribution parameters in the `simulate_experiment` function to avoid bias towards lower success probability.

The current specification of the beta distribution:

```
theta = np.random.beta(conversions + 1, exposures + 1)
``` 
treats every exposure as a failure, that is overstates the failures thus undervalues the success probabilities of the variations. The effect is pronounced for variations with very high baseline conversion rates but less severe for variations with extremely low conversion rates. 

Traditionally, the Thompson Sampling Algorithm for the Bernoulli Bandit is:


```math
\begin{align*}
1: & \text{for } t = 1, 2, \ldots \text{ do:} \\
2: & \quad \quad \text{Sample model:} \\
3: & \quad \quad \text{for } k = 1 \text{ to } K \text{ do:} \\
4: & \quad \quad \quad \text{Sample } \theta_k \sim \text{beta}(\alpha_k, \beta_k) \\
5: & \quad \quad \text{$$end for$$} \\
6: \\
7: & \quad \quad \text{Select and apply action:} \\
8: & \quad \quad x_t \leftarrow argmax_k  \theta_k \\
9: & \quad \quad \text{Apply } x_t \text{ and observe } r_t \\
10: \\
11: & \quad \quad \text{Update distribution:} \\
12: & \quad \quad (\alpha_{x_t}, \beta_{x_t}) \leftarrow (\alpha_{x_t} + r_t, \beta_{x_t} + 1 - r_t) \\
13: & \text{end for}
\end{align*}
```
Where  α, β represent the parameters of each arm i.e. the success and failure counts, respectively OR the number of `conversions` and `non-conversions`, respectively. 

```
non-conversions (or beta)  = exposures - conversions
```
@james-jory james-jory self-requested a review October 4, 2024 18:09
Copy link
Contributor

@james-jory james-jory left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for catching this oversight!

@james-jory james-jory merged commit 01f9d3d into aws-samples:master Oct 4, 2024
2 checks passed
MustaphaU added a commit to MustaphaU/retail-demo-store that referenced this pull request Oct 4, 2024
…ndex` method

Update the beta distribution parameters in the `_select_variation_index` method to avoid bias towards lower success probability.

The current specification of the beta distribution:

```
theta = np.random.beta(conversions + 1, exposures + 1)
``` 
treats every exposure as a failure, that is overstates the failures thus undervalues the success probabilities of the variations. The effect is pronounced for variations with very high baseline conversion rates but less severe for variations with extremely low conversion rates. 

aws-samples#643
@MustaphaU
Copy link
Contributor Author

Thank you for catching this oversight!

Thank you @james-jory !
#647

james-jory added a commit that referenced this pull request Oct 28, 2024
…ndex` method (#647)

Update the beta distribution parameters in the `_select_variation_index` method to avoid bias towards lower success probability.

The current specification of the beta distribution:

```
theta = np.random.beta(conversions + 1, exposures + 1)
``` 
treats every exposure as a failure, that is overstates the failures thus undervalues the success probabilities of the variations. The effect is pronounced for variations with very high baseline conversion rates but less severe for variations with extremely low conversion rates. 

#643

Co-authored-by: James Jory <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Shipped
Development

Successfully merging this pull request may close these issues.

2 participants