pymc-marketing icon indicating copy to clipboard operation
pymc-marketing copied to clipboard

Implement validation PPC plots for CLV models

Open ricardoV94 opened this issue 1 year ago • 1 comments

https://lifetimes.readthedocs.io/en/latest/Quickstart.html

TODO:

  • Find out what metrics we want to do PPC wrt, implement plots
  • Check if there are model-specific ones as an extra (look at lifetimes for examples)

ricardoV94 avatar May 03 '23 13:05 ricardoV94

I seem to have created a duplicate issue for this: https://github.com/pymc-labs/pymc-marketing/issues/326

This is highly relevant to the work I'm doing in https://github.com/pymc-labs/pymc-marketing/pull/327. The dev notebook in that branch even has some plots that can be used for both prior and posterior predictive checks for purchase frequency PMFs:

with pnbd.model:
    post_idata = pm.sample_posterior_predictive(pnbd.idata)

obs_freq = post_idata.observed_data['likelihood'][...,1].to_pandas().value_counts().sort_index()
post_pred_freq = post_idata.posterior_predictive['likelihood'][0][0][...,1].to_pandas().value_counts().sort_index()

plot_posterior_pred = pd.DataFrame({"posterior_predictive": post_pred_freq, "observed": obs_freq})

plot_posterior_pred.plot(kind="bar", title = "Plot Post Predictive")

image

plt.style.use('arviz-darkgrid')

obs_freq = post_idata.observed_data['likelihood'][...,1]
post_pred_freq = post_idata.posterior_predictive['likelihood'][0][0][...,1]
az.plot_ecdf(post_pred_freq, obs_freq, confidence_bands = True).set_title("ECDF")

image

az.plot_ecdf(post_pred_freq, obs_freq, confidence_bands = True, difference=True)

image

ColtAllen avatar Jul 18 '23 17:07 ColtAllen