pymc-marketing icon indicating copy to clipboard operation
pymc-marketing copied to clipboard

Add `log-likelihood` Attribute to fitted `InferenceData` object

Open ColtAllen opened this issue 2 years ago • 6 comments
trafficstars

The Widely Available Information Criterion (WAIC) is a useful model comparison metric:

https://python.arviz.org/en/stable/api/generated/arviz.waic.html

However, the InferenceData object returned from CLV and MMM models do not contain the required log-likelihood group.

ColtAllen avatar Aug 09 '23 18:08 ColtAllen

For MCMC fits this can be done via sample_config:

mcmc_fit = ParetoNBDModel(df,sampler_config = {"idata_kwargs": {"log_likelihood": True}})

WAIC doesn't really apply for MAP fits though.

ColtAllen avatar Nov 10 '23 18:11 ColtAllen

We can call pm.compute_log_likelihood for methods that need this quantity when it's missing?

https://www.pymc.io/projects/docs/en/latest/api/generated/pymc.compute_log_likelihood.html#pymc.compute_log_likelihood

The critical thing is we need to add Potentials manually to var_names. This means this venue is actually the best because the default through the idata_kwargs doesn't allow you to specify it AFAIK

ricardoV94 avatar Nov 11 '23 11:11 ricardoV94

Actually that method checks the variable is an observed_RV, we should lift the restriction to allow Potentials as well. The value transform logic in that method can also be removed in favor of https://www.pymc.io/projects/docs/en/latest/api/model/generated/pymc.model.transform.conditioning.remove_value_transforms.html

ricardoV94 avatar Nov 11 '23 11:11 ricardoV94

BetaGeoModel is currently using a Potential, which precludes the use of an important evaluation function. If this were implemented along with a data simulation method, I wonder if it would be more performant than the external distribution block approach used in ParetoNBDModel.

ColtAllen avatar Feb 24 '24 18:02 ColtAllen

I wonder if it would be more performant than the external distribution block approach used in ParetoNBDModel.

That shouldn't matter

ricardoV94 avatar Feb 24 '24 19:02 ricardoV94

Could be good with #742

williambdean avatar Jun 19 '24 06:06 williambdean