2.) List Scoring | Sophiebrandeis

Blue Apron is a subscription based meal delivery service that provides subscribers with three meal kits per week for 52 weeks of the year. In this problem set, we call Blue Apron, “Orange Apron” and are tasked with advising them as they are considering a customer acquisition campaign and want to run a field experiment to determine the appropriate target customers. To implement the campaign, Blue Apron has rented a list containing information on 500 households. The list contains some information about the households captured in four variables. Here is a Sample of the dataset:

About the dataset:

The first variable is a binary indicator of whether children are present in the household (1=yes, 0=no).

The remaining variables are three “hotline” buying indices. The three hotline indices in this dataset are listed as h1, h2 and h3.

*Orange Apron sent an invitation to all 500 names on the list to join service. The invitation offer includes a deep discount on three weeks of service. We observe whether or not

each of the 500 consumers accepted the invitation:

The value of y is 1 if the person joined the service and the value is 0 otherwise.

* The dataset is then broken up into an Estimation List and a Holdout List:

Estimation-list: A random sample of 244 persons as the estimation sample (i.e., to estimate the scoring model with this data).
Holdout-list: The second list of 256 is used to test list scoring and evaluate how successful/accurate the moel that selcts the targets was.

Here, I ran a logistic regression model to predict an individual’s decision to join the meal delivery service as a function of the scoring variables (children/no children, and all 3 hotline variables).
- My results show that the variables having children and h1 have a positive correlation with purchasing Orange Apron, while the h2 and h3 variables have a negative correlation . This can be explained by looking at the beta coefficients and their correlations to the intercept (join or no join). H2 and 3 are negatively correlated so they have a negative intercept, while children and h1 are positively correlated. However, H3's P value is greater than .05 so it is not statistically significant. This can also be seen through h3 only having a .004 correlation with the intercept.
- A 1-unit increase in the HL1 has a 3.3% increase in the odds of joining the service.
- A 1-unit increase in the HL2 has a 2.7% decrease in the odds of joining the service.
- A 1-unit increase in the HL3 has a .4% decrease in the odds of joining the service, but is statistically insignificant meaning that Orange Apron does not need to take this factor into account when deciding who to target.

Using the estimation dataset, I computed the score (higher the score, more promising the consumer is to target) for all individuals in the holdout data.
Then using that calculated score, I computed (for each individual): the predicted response rate and the resulting lift (a number representing an increase in sales as a result of the targeting).
I also computed the marginal effect of 1 unit change in each of the hotline variables on the response probabilities.
Below are my reported average response rate, lift and marginal effects computed over all 256 holdout individuals:

I sorted the hold-out list in decreasing order of response probability and plotted the expected and actual sales from sending 1-256 solicitations.

This shows that the model's predicative performance is accurate, and we can use it as a relatively reliable model. Around the 100 persons mark the model under predicts a bit and towards the end at around 200, it slightly over predicts.

The grocery and meal delivery service is notorious for low margins and high customer churn rates. In this project, Orange Apron predicts the average customer lifetime value (CLV) to be $13.50.
I assumed solicitation costs were $3.00 and that the list owner charged a cost of $1.00/ household and determined the cut-off probability for making offers using the marginal cost rule.

Findings: I found that with targeting, 69.14% of the hold-out clients will be targeted based on their recorded information, which takes profits up to $63.50.
By targeting Orange Apron is able to make a positive profit margin ($63.50) because they are focusing their sales to those who are likely to buy. By not targeting, Orange Apron will face a negative profit margin of -$25, this is because they are targeting everyone and not taking variables and demographics into account.