lightweight_mmm icon indicating copy to clipboard operation
lightweight_mmm copied to clipboard

how to treat zero cost channels and attribution hints

Open chanansh opened this issue 1 year ago • 6 comments

I have a data frame which looks like this: date | channel name | cost | leads

Some of the channels' cost is zero. The number of leads per channel is based on a simple attribution like first or last touch. In MMM usually one just counts the total outcome, e.g. df.groupby('date').leads.sum() however, I feel some information is lost. Moreover, how should I treat zero-cost channels? The ROI is infinite.

chanansh avatar Aug 25 '23 09:08 chanansh

As the "cost" for zero-cost channels, for instance organic, I have passed in numbers such as "total visitors" in the past, essentially in LightweightMMM the media data itself. Then it becomes a measure of how driving visitors, leads to "target". The overall ROI of visitors. So I can compare my organic channels to eachother, and my paid media channels to eachother.

becksimpson avatar Sep 08 '23 10:09 becksimpson

Can you pass zero-cost channels like organic as control variables instead?

Aanai avatar Oct 27 '23 07:10 Aanai

@Aanai You can, they would not be modelled with saturation effects or adstock, just with a linear multiplier. That is the only tradeoff. If you find that they learn negative coefficients when the model should not, and all your extra_features channels should have strictly non-zero effects, you can change the prior distribution in models.py for _EXTRA_FEATURES, to halfnormal, from normal, this will prevent negative attribution. Line 106. _COEF_EXTRA_FEATURES: dist.Normal(loc=0., scale=1.), _COEF_EXTRA_FEATURES: dist.HalfNormal(scale=1.),

becksimpson avatar Oct 27 '23 10:10 becksimpson

Thank you. I want only one of the threeextra_features to be strictly non-zero. Is it not possible to pass a custom prior for just one coef_extra_features ?

Aanai avatar Nov 03 '23 04:11 Aanai

@Aanai I think you can accomplish this with with a TruncateNormalDistribution put into custom_priors. Say your third extra feature is strictly non-zero.

custom_priors = {
_COEF_EXTRA_FEATURES: numpyro.distributions.TruncatedNormalDistribution(
    loc = jnp.zeros((3,)),
    scale=jnp.ones((3,)), 
    low=jnp.array([-5, -5, 0])
)
}

becksimpson avatar Nov 07 '23 09:11 becksimpson

@becksimpson hey there, thank you for your input in this thread, I've come to some good insights, thank you! I was wondering if it's possible to perform the same process you mentioned to @Aanai, but for paid media channels, those we should use in 'lag_weight' custom prior, which uses a beta distribution. Let's say I have 4 paid media channels, how could I customize each distribution of each channel in my custom prior?

Or otherwise, suppose I did not define any custom prior, how can I get the trace for lag_weight of each of my channels? Would this work?

mmm= lightweight_mmm.LightweightMMM(model_name="adstock")
mmm.fit(...)
mmm._mcmc.get_samples()['lag_weight']

cremerf avatar Mar 12 '24 18:03 cremerf