lightweight_mmm
lightweight_mmm copied to clipboard
Is it normal for the baseline contribution to hit zero more often than being above 0?
Oftentimes I'm finding that the baseline contribution runs down to zero no matter what features or timeline I use for the data. Can anyone help me out on why this is happening and/or if this is normal behavior when you have a large number of media channels?
You can control this to some degree by tightening and lowering your media priors; if you have a lot of channels you need to decrease the prior per channel.
I'm seeing the same behavior, and I will say that reducing the media priors makes this less of a problem, but then I feel like I'm setting priors based on how I want to posterior to look and not my actual prior belief. I'm looking into setting the media priors based on something other than cost as a result.
@lorenze3 Please let me know if you come up with anything! I found that setting more informed priors (namely using the below custom_priors
) to shrink the adstock lag_weight
parameter for one of my channels was somehow a success for it, as well as using a different seed. Not sure how the seed can impact it that much, but changing it from 105 to 12 worked for me for the most part.
c1 = np.array([2] * len(media_cols))
c1[7] = 1.5
c0 = np.ones(len(media_cols))
custom_priors = {'lag_weight': {'concentration1': c1, 'concentration0': c0}}
@steven-struglia -- thanks for the tip. I'm a bit attached to a lag_weight of .5 as a 'natural' choice, so I set both c1 and c0 to 1. I might move off of that -- especially spreading out the prior a bit, once I wrap my head around the beta pdf a bit more better -- but for now I like the weight results I'm getting.
As for this :
Not sure how the seed can impact it that much, but changing it from 105 to 12 worked for me for the most part. I'm wondering about your sampling choices -- warmup, number of iterations, and number of chains? The seed sets the sampling, so I guess you could get real (un)lucky with your choice, but I'd love to think the MCMC sampling wouldn't converge to different places for most seeds . . . (this is when I say I have very little bayesian analysis experience, but to my understanding the seed shouldn't be very influential for most seed choices).
@steven-struglia -- I really want to check on what happens if we 'warm start' adstock by having a pre-model period for the media data, but in the meantime I've been able to adjust out a lot of my disappearing core behavior by reducing the range on the expo_trend prior -- previously, the mean was coming in between .5 and .7, now I'm offering a prior (it's default is uniform distribution) between .9 and 1.1, the mean of the posterior is coming in around 1, and base volume is non zero everywhere. . .