lightweight_mmm icon indicating copy to clipboard operation
lightweight_mmm copied to clipboard

Is it normal for the baseline contribution to hit zero more often than being above 0?

Open steven-struglia opened this issue 2 years ago • 5 comments

Oftentimes I'm finding that the baseline contribution runs down to zero no matter what features or timeline I use for the data. Can anyone help me out on why this is happening and/or if this is normal behavior when you have a large number of media channels?

image

steven-struglia avatar Jan 07 '23 00:01 steven-struglia

You can control this to some degree by tightening and lowering your media priors; if you have a lot of channels you need to decrease the prior per channel.

michevan avatar Jan 10 '23 21:01 michevan

I'm seeing the same behavior, and I will say that reducing the media priors makes this less of a problem, but then I feel like I'm setting priors based on how I want to posterior to look and not my actual prior belief. I'm looking into setting the media priors based on something other than cost as a result.

lorenze3 avatar Jan 11 '23 18:01 lorenze3

@lorenze3 Please let me know if you come up with anything! I found that setting more informed priors (namely using the below custom_priors) to shrink the adstock lag_weight parameter for one of my channels was somehow a success for it, as well as using a different seed. Not sure how the seed can impact it that much, but changing it from 105 to 12 worked for me for the most part.

c1 = np.array([2] * len(media_cols))
c1[7] = 1.5
c0 = np.ones(len(media_cols))
custom_priors = {'lag_weight': {'concentration1': c1, 'concentration0': c0}}

steven-struglia avatar Jan 11 '23 19:01 steven-struglia

@steven-struglia -- thanks for the tip. I'm a bit attached to a lag_weight of .5 as a 'natural' choice, so I set both c1 and c0 to 1. I might move off of that -- especially spreading out the prior a bit, once I wrap my head around the beta pdf a bit more better -- but for now I like the weight results I'm getting.

As for this :

Not sure how the seed can impact it that much, but changing it from 105 to 12 worked for me for the most part. I'm wondering about your sampling choices -- warmup, number of iterations, and number of chains? The seed sets the sampling, so I guess you could get real (un)lucky with your choice, but I'd love to think the MCMC sampling wouldn't converge to different places for most seeds . . . (this is when I say I have very little bayesian analysis experience, but to my understanding the seed shouldn't be very influential for most seed choices).

lorenze3 avatar Jan 11 '23 21:01 lorenze3

@steven-struglia -- I really want to check on what happens if we 'warm start' adstock by having a pre-model period for the media data, but in the meantime I've been able to adjust out a lot of my disappearing core behavior by reducing the range on the expo_trend prior -- previously, the mean was coming in between .5 and .7, now I'm offering a prior (it's default is uniform distribution) between .9 and 1.1, the mean of the posterior is coming in around 1, and base volume is non zero everywhere. . .

lorenze3 avatar Jan 16 '23 16:01 lorenze3