probability
probability copied to clipboard
Any plans to update the examples to TensorFlow 2?
The examples in https://github.com/tensorflow/probability/tree/master/tensorflow_probability/examples do not run with TensorFlow 2, but it's been a while the stable TF2 is available. So, any plans to update the examples to TensorFlow 2 (eager execution)? I am particularly interested in https://github.com/tensorflow/probability/blob/master/tensorflow_probability/examples/bayesian_neural_network.py.
Even I am interested in TF2.0 examples.
I am interested in giving this a go. Can I be assigned to work on it?
@Pyrsos Well, if you have a go at it, share it with us, please! However, note that there are a few bugs, which may prevent you from updating certain examples to TF2. See https://github.com/tensorflow/probability/issues/620.
Is there any timeline on this or has anyone played with any examples ?
We have taken a pass over the examples to assess what work might need to be done (which might be a bit out of date, as @davmre has already made some updates). If anyone wants to take some of these on, please feel free to send PRs, esp for the easy-ish ones!
File | Purpose | Showcases | What needs work | Proposal |
---|---|---|---|---|
tensorflow_probability/examples/bayesian_neural_network.py | Trains a Bayesian neural network to classify MNIST digits. | tfp.layers.Convolution2DFlipouttfp.layers.DenseFlipoutinline in a keras modelCategorical output (but not tfpl.DistributionLambda) | TF2 (placeholder, session)tf.compat.v1.metrics.accuracytf.compat.v1.data.make_one_shot_iterator | |
tensorflow_probability/examples/cifar10_bnn.py | Trains a Bayesian neural network to classify CIFAR-10 images. | tfd.Categorical(+bayesian_{vgg,resnet}) | TF2 (placeholder, session)tf.compat.v1.metrics.accuracymigrate to colab?uses tensorflow_probability.examples.models.* | Easy to update |
tensorflow_probability/examples/deep_exponential_family.py | Trains a sparse Gamma deep exponential family on NIPS 2011 conference papers. | tfp.edward2 (could be JDCoroutine) | tf.compat.v1 variables, summary, global stepuses edward2Looks like great opp to showcase JDCoroutineUses NIPS2011 papers dataset (migrate to TFDS?) | Delete or rewrite |
tensorflow_probability/examples/disentangled_vae.py | Disentangled Sequential Variational Autoencoder. | Uses a handful of distributions- Independent(Normal, 3) images- MVNDiag | Written in eager, so easier TF2 migrationuses sprites_datasettf.compat.v1.train.cosine_decayuses tf.contrib.checkpointLearnableMultivariateNormalDiag can probably use TF2 var safety work | Delete or rewrite |
tensorflow_probability/examples/disentangled_vae_test.py | test for dvae | tf.compat.v1.global_variables_initializer() | ||
tensorflow_probability/examples/generative_adversarial_network.py | Trains a GAN to generate synthetic images of MNIST digits. | n/a (Bernoulli!) | Connection to TFP is tenuous (tfd.Bernoulli), an extremely basic GAN | Delete |
tensorflow_probability/examples/grammar_vae.py | Trains a grammar variational auto-encoder on synthetic data. | Works on molecular representations (SMILES strings)Ed as universal PPL (no batching), uses tracer for branchy model | Uses edward2, ed.tapetf.compat.v1.nn.rnn_cell.LSTMCell (would need to -> keras RNN) | Delete (or figure out how one would write with TFP!) |
tensorflow_probability/examples/latent_dirichlet_allocation_distributions.py | Trains a Latent Dirichlet Allocation (LDA) model on 20 Newsgroups. | Dirichlet, OneHotCategorical, kl_divergence | uses tf.compat.v1:.summary.get_or_create_global_step.metrics.mean.py_func.data.make_one_shot_iteratorEstimatorDoesn't use JD (merge w/ lda_ed2) | Delete or rewrite |
tensorflow_probability/examples/latent_dirichlet_allocation_edward2.py | Trains a Latent Dirichlet Allocation (LDA) model on 20 Newsgroups. | Easily rewritten w/ JDCoro (merge w/ lda_dists) | Uses edward2 (12 lines shorter than the above) | Delete |
tensorflow_probability/examples/logistic_regression.py | Trains a Bayesian logistic regression model on synthetic data. | tfp.layers.DenseFlipouttfd.Bernoulli | - Uses toy data (could we use covertype?)- tfp.layers.DenseFlipout- make_one_shot_iterator- plotting in shell script, why not colab?- small, easy upgrade | Update |
tensorflow_probability/examples/models/bayesian_resnet.py | Constructs a ResNet18 model. | tfp.layers.Convolution2DFlipouttfp.layers.DenseFlipout(inline in deep keras model) | Uses tfp.layers.DenseFlipout (needs audit for TF2) | Easy to keep |
tensorflow_probability/examples/models/bayesian_vgg.py | Constructs a VGG16 model. | tfp.layers.Convolution2DFlipouttfp.layers.DenseFlipout(inline in deep keras model) | Uses tfp.layers.DenseFlipout (needs audit for TF2) | Easy to keep |
tensorflow_probability/examples/README.md | This directory contains standalone usage examples of TensorFlow Probability API. | Update to reflect decisions | Update readme | |
tensorflow_probability/examples/sprites_dataset.py | Liberated Pixel Cup [(LPC)][1] Sprites Dataset. | n/a | Uses tf.contrib.lookup.index_table_from_tensor | Move to tfds, or delete |
tensorflow_probability/examples/vae.py | Trains a variational auto-encoder (VAE) on binarized MNIST. | MultivariateNormalDiagIndep(Bern)MixSameFam(MVNDiag)kl_divergence | - upgrade to tfds binarized_mnist- tf.compat.v1: - py_func - make_one_shot_iterator - summary - metrics - train.cosine_decay- Estimator- MoG prior, MVNDiag latents | Delete or rewrite, I think we would do this differently today |
tensorflow_probability/examples/vq_vae.py | Trains a vector quantized-variational autoencoder (VQ-VAE) on MNIST. | Indep(Bern)Multinomial | Uses moving_averages direct TF import.Three variants of MNIST - are they all in TFDS?tf.compat.v1.data.Iterator.from_string_handlepy_func (for dataset parsing)tf.compat.v1.summaryA lot of plotting --> colab?TF2 (Graph/Session) | Rewrite, VQVAE is (imo) an important example to have |
tensorflow_probability/examples/jupyter_notebooks/A_Tour_of_TensorFlow_Probability.ipynb | On the label | Tons of stuff (dists, bijectors, psd kernels, gp regression, hmc, step size adapt, etc) | tfp.positive_semidefinite_kernelsmay need to update for mcmc changes when those landmaybe drop the tfp-nightly picker and the tf2/tf1 picker? | Update |
tensorflow_probability/examples/jupyter_notebooks/Bayesian_Gaussian_Mixture_Model.ipynb | On the label | dists, bijectors, hmc, ttk | MVNCholPrecisionTriL should move to tfp proper?Needs TF2 updatesuggest update to JointDist as wellupdate unconstraining_bijectors naming? | Update |
tensorflow_probability/examples/jupyter_notebooks/Bayesian_Switchpoint_Analysis.ipynb | On the label (mining disasters dataset) | ed2 | Update to tfp101 changepoint model?ed2 -> JD?Needs TF2 updateuses legacy step size adaptuse log_rate for poisson? | Delete or update (update not too hard) |
tensorflow_probability/examples/jupyter_notebooks/Credit_Card_Interest.ipynb | Some guessing about whether interest rates go up, simulating possible futures | tfd.Binomial | Too trivial | Delete |
tensorflow_probability/examples/jupyter_notebooks/Eight_Schools.ipynb | Classic hierarchical | ed2 | Needs TF2 update, ed2->JD update | updated in cl/278789359 |
tensorflow_probability/examples/jupyter_notebooks/Factorial_Mixture.ipynb | Nifty little demo/trick | batching, MixSameFam | Good demo case for tape safety, could use quick TF2 audit, already eager. | Update |
tensorflow_probability/examples/jupyter_notebooks/Fitting_DPMM_Using_pSGLD.ipynb | Showcases some synergies of TF+TFP | SGLD optimizer, transformations of tf.Variable | Needs the colab link badgeNeeds TF2 updateNice showcase for tfp.util.TransformedVariable, tape-safety | Update |
tensorflow_probability/examples/jupyter_notebooks/Gaussian_Copula.ipynb | Nifty demo/trick | bijectors, transformed dist | Concat bijector can become tfb.Blockwise?Needs TF2 updateShowcase other CDF bijectors? Off-hand I know we have Weibull. | Update |
tensorflow_probability/examples/jupyter_notebooks/Gaussian_Process_Latent_Variable_Model.ipynb | Demo the GP library w/ MNIST | GP, GPRM | tfp.positive_semidefinite_kernelsNeeds TF2 update, doesn't look very hard | Update |
tensorflow_probability/examples/jupyter_notebooks/Gaussian_Process_Regression_In_TFP.ipynb | Kind of classic GP demo | GP, GPRM | tfp.positive_semidefinite_kernelsNeeds TF2 update, doesn't look very hardUses AdamOptimizer, so loss needs to become a function, probably tf.function | updated in cl/279842677 |
tensorflow_probability/examples/jupyter_notebooks/Generalized_Linear_Models.ipynb | Explain GLMs, demo the tfp.glm library | GLM | Needs TF2 update, ed2->JD updateI don't expect a user to be able to execute the R section (suggest rpy2, https://colab.research.google.com/drive/1h5-lqnPz3WRXLXIt41N5B_aDngIIlfyL prototype, but note the r install took FOREVER for some reason)Uses tf.gradients at the bottom, consider removing that section unless we really value the didactic nature. | Update |
tensorflow_probability/examples/jupyter_notebooks/HLM_TFP_R_Stan.ipynb | Comparing TFP (SA-EM), R, Stan for the radon dataset | Hierarchical model | Similar to GLMs.ipynb re: rpy2Needs TF2 migrationShould probably switch to JDAppears to autodiff through HMC? Is this still supported? | Update or delete |
tensorflow_probability/examples/jupyter_notebooks/Linear_Mixed_Effects_Model_Variational_Inference.ipynb | VI on radon dataset | dists, autodiff, trainable surrogate posterior, tfp.vi.monte_carlo_csiszar_f_divergence, tfp.vi.kl_reverse | tf.make_templateuse JDTF2 migration | Update or delete |
tensorflow_probability/examples/jupyter_notebooks/Linear_Mixed_Effects_Models.ipynb | InstEval dataset from lme4 | dists, autodiff, expectation maximization | Needs TF2 migration | Updated in cl/282628627 |
tensorflow_probability/examples/jupyter_notebooks/Modeling_with_JointDistribution.ipynb | JDSeq | JD, HMC, batching, L-BFGS, VI, multiple datasets, MF VI, TriL VI | Includes GLMM on radon, can we delete the LME notebooks?tfd.fill_triangular TODO(junpenglao): update to use NUTS, also clean up VI part. | updated in cl/279789837 |
tensorflow_probability/examples/jupyter_notebooks/Multilevel_Modeling_Primer.ipynb | Similar to HLM, LMEMVI, LMEM, GLM - nice graphics and renderings of the graphical models | JD, deeper dive, and newer than prev GLMM notebooks | Already TF2! | Keep |
tensorflow_probability/examples/jupyter_notebooks/Multiple_changepoint_detection_and_Bayesian_model_selection.ipynb | Multiple changepoint inhomogenous poisson process, using HMM | tfd.HMM, MAP estimation, autodiff | TF2 migrationTODO(davmre): once tfp.HiddenMarkovModel implements the Viterbi algorithm to find highest-probability trajectories, update this section to use it. | Update |
tensorflow_probability/examples/jupyter_notebooks/Optimizers_in_TensorFlow_Probability.ipynb | ||||
tensorflow_probability/examples/jupyter_notebooks/Probabilistic_Layers_Regression.ipynb | ||||
tensorflow_probability/examples/jupyter_notebooks/Probabilistic_Layers_VAE.ipynb | ||||
tensorflow_probability/examples/jupyter_notebooks/Probabilistic_PCA.ipynb | updated in cl/278789359 | |||
tensorflow_probability/examples/jupyter_notebooks/Structural_Time_Series_Modeling_Case_Studies_Atmospheric_CO2_and_Electricity_Demand.ipynb | bsts, vi interface | tfp.sts, tfp.vi | Already TF2! | Keep |
tensorflow_probability/examples/jupyter_notebooks/TensorFlow_Distributions_Tutorial.ipynb | ||||
tensorflow_probability/examples/jupyter_notebooks/TensorFlow_Probability_Case_Study_Covariance_Estimation.ipynb | ||||
tensorflow_probability/examples/jupyter_notebooks/Understanding_TensorFlow_Distributions_Shapes.ipynb |
In addition to the example notebooks etc. mentioned above, the doc comments also have v1 code. For instance in tfp.distributions.Mixture
where the main doc comment has an example using .eval()
which is not needed in TF2 where eager execution is default.
is this issue still open for work?
is this issue resolved or still needs to be fixed ?