probability icon indicating copy to clipboard operation
probability copied to clipboard

Any plans to update the examples to TensorFlow 2?

Open nbro opened this issue 5 years ago • 8 comments

The examples in https://github.com/tensorflow/probability/tree/master/tensorflow_probability/examples do not run with TensorFlow 2, but it's been a while the stable TF2 is available. So, any plans to update the examples to TensorFlow 2 (eager execution)? I am particularly interested in https://github.com/tensorflow/probability/blob/master/tensorflow_probability/examples/bayesian_neural_network.py.

nbro avatar Oct 14 '19 15:10 nbro

Even I am interested in TF2.0 examples.

shashankg7 avatar Oct 16 '19 03:10 shashankg7

I am interested in giving this a go. Can I be assigned to work on it?

Pyrsos avatar Nov 07 '19 09:11 Pyrsos

@Pyrsos Well, if you have a go at it, share it with us, please! However, note that there are a few bugs, which may prevent you from updating certain examples to TF2. See https://github.com/tensorflow/probability/issues/620.

nbro avatar Nov 07 '19 14:11 nbro

Is there any timeline on this or has anyone played with any examples ?

patelmiteshn avatar Dec 09 '19 23:12 patelmiteshn

We have taken a pass over the examples to assess what work might need to be done (which might be a bit out of date, as @davmre has already made some updates). If anyone wants to take some of these on, please feel free to send PRs, esp for the easy-ish ones!

File Purpose Showcases What needs work Proposal
tensorflow_probability/examples/bayesian_neural_network.py Trains a Bayesian neural network to classify MNIST digits. tfp.layers.Convolution2DFlipouttfp.layers.DenseFlipoutinline in a keras modelCategorical output (but not tfpl.DistributionLambda) TF2 (placeholder, session)tf.compat.v1.metrics.accuracytf.compat.v1.data.make_one_shot_iterator  
tensorflow_probability/examples/cifar10_bnn.py Trains a Bayesian neural network to classify CIFAR-10 images. tfd.Categorical(+bayesian_{vgg,resnet}) TF2 (placeholder, session)tf.compat.v1.metrics.accuracymigrate to colab?uses tensorflow_probability.examples.models.* Easy to update
tensorflow_probability/examples/deep_exponential_family.py Trains a sparse Gamma deep exponential family on NIPS 2011 conference papers. tfp.edward2 (could be JDCoroutine) tf.compat.v1 variables, summary, global stepuses edward2Looks like great opp to showcase JDCoroutineUses NIPS2011 papers dataset (migrate to TFDS?) Delete or rewrite
tensorflow_probability/examples/disentangled_vae.py Disentangled Sequential Variational Autoencoder. Uses a handful of distributions- Independent(Normal, 3) images- MVNDiag Written in eager, so easier TF2 migrationuses sprites_datasettf.compat.v1.train.cosine_decayuses tf.contrib.checkpointLearnableMultivariateNormalDiag can probably use TF2 var safety work Delete or rewrite
tensorflow_probability/examples/disentangled_vae_test.py test for dvae   tf.compat.v1.global_variables_initializer()  
tensorflow_probability/examples/generative_adversarial_network.py Trains a GAN to generate synthetic images of MNIST digits. n/a (Bernoulli!) Connection to TFP is tenuous (tfd.Bernoulli), an extremely basic GAN Delete
tensorflow_probability/examples/grammar_vae.py Trains a grammar variational auto-encoder on synthetic data. Works on molecular representations (SMILES strings)Ed as universal PPL (no batching), uses tracer for branchy model Uses edward2, ed.tapetf.compat.v1.nn.rnn_cell.LSTMCell (would need to -> keras RNN) Delete (or figure out how one would write with TFP!)
tensorflow_probability/examples/latent_dirichlet_allocation_distributions.py Trains a Latent Dirichlet Allocation (LDA) model on 20 Newsgroups. Dirichlet, OneHotCategorical, kl_divergence uses tf.compat.v1:.summary.get_or_create_global_step.metrics.mean.py_func.data.make_one_shot_iteratorEstimatorDoesn't use JD (merge w/ lda_ed2) Delete or rewrite
tensorflow_probability/examples/latent_dirichlet_allocation_edward2.py Trains a Latent Dirichlet Allocation (LDA) model on 20 Newsgroups. Easily rewritten w/ JDCoro (merge w/ lda_dists) Uses edward2 (12 lines shorter than the above) Delete
tensorflow_probability/examples/logistic_regression.py Trains a Bayesian logistic regression model on synthetic data. tfp.layers.DenseFlipouttfd.Bernoulli - Uses toy data (could we use covertype?)- tfp.layers.DenseFlipout- make_one_shot_iterator- plotting in shell script, why not colab?- small, easy upgrade Update
tensorflow_probability/examples/models/bayesian_resnet.py Constructs a ResNet18 model. tfp.layers.Convolution2DFlipouttfp.layers.DenseFlipout(inline in deep keras model) Uses tfp.layers.DenseFlipout (needs audit for TF2) Easy to keep
tensorflow_probability/examples/models/bayesian_vgg.py Constructs a VGG16 model. tfp.layers.Convolution2DFlipouttfp.layers.DenseFlipout(inline in deep keras model) Uses tfp.layers.DenseFlipout (needs audit for TF2) Easy to keep
tensorflow_probability/examples/README.md This directory contains standalone usage examples of TensorFlow Probability API.   Update to reflect decisions Update readme
tensorflow_probability/examples/sprites_dataset.py Liberated Pixel Cup [(LPC)][1] Sprites Dataset. n/a Uses tf.contrib.lookup.index_table_from_tensor Move to tfds, or delete
tensorflow_probability/examples/vae.py Trains a variational auto-encoder (VAE) on binarized MNIST. MultivariateNormalDiagIndep(Bern)MixSameFam(MVNDiag)kl_divergence - upgrade to tfds binarized_mnist- tf.compat.v1: - py_func - make_one_shot_iterator - summary - metrics - train.cosine_decay- Estimator- MoG prior, MVNDiag latents Delete or rewrite, I think we would do this differently today
tensorflow_probability/examples/vq_vae.py Trains a vector quantized-variational autoencoder (VQ-VAE) on MNIST. Indep(Bern)Multinomial Uses moving_averages direct TF import.Three variants of MNIST - are they all in TFDS?tf.compat.v1.data.Iterator.from_string_handlepy_func (for dataset parsing)tf.compat.v1.summaryA lot of plotting --> colab?TF2 (Graph/Session) Rewrite, VQVAE is (imo) an important example to have
tensorflow_probability/examples/jupyter_notebooks/A_Tour_of_TensorFlow_Probability.ipynb On the label Tons of stuff (dists, bijectors, psd kernels, gp regression, hmc, step size adapt, etc) tfp.positive_semidefinite_kernelsmay need to update for mcmc changes when those landmaybe drop the tfp-nightly picker and the tf2/tf1 picker? Update
tensorflow_probability/examples/jupyter_notebooks/Bayesian_Gaussian_Mixture_Model.ipynb On the label dists, bijectors, hmc, ttk MVNCholPrecisionTriL should move to tfp proper?Needs TF2 updatesuggest update to JointDist as wellupdate unconstraining_bijectors naming? Update
tensorflow_probability/examples/jupyter_notebooks/Bayesian_Switchpoint_Analysis.ipynb On the label (mining disasters dataset) ed2 Update to tfp101 changepoint model?ed2 -> JD?Needs TF2 updateuses legacy step size adaptuse log_rate for poisson? Delete or update (update not too hard)
tensorflow_probability/examples/jupyter_notebooks/Credit_Card_Interest.ipynb Some guessing about whether interest rates go up, simulating possible futures tfd.Binomial Too trivial Delete
tensorflow_probability/examples/jupyter_notebooks/Eight_Schools.ipynb Classic hierarchical ed2 Needs TF2 update, ed2->JD update updated in cl/278789359
tensorflow_probability/examples/jupyter_notebooks/Factorial_Mixture.ipynb Nifty little demo/trick batching, MixSameFam Good demo case for tape safety, could use quick TF2 audit, already eager. Update
tensorflow_probability/examples/jupyter_notebooks/Fitting_DPMM_Using_pSGLD.ipynb Showcases some synergies of TF+TFP SGLD optimizer, transformations of tf.Variable Needs the colab link badgeNeeds TF2 updateNice showcase for tfp.util.TransformedVariable, tape-safety Update
tensorflow_probability/examples/jupyter_notebooks/Gaussian_Copula.ipynb Nifty demo/trick bijectors, transformed dist Concat bijector can become tfb.Blockwise?Needs TF2 updateShowcase other CDF bijectors? Off-hand I know we have Weibull. Update
tensorflow_probability/examples/jupyter_notebooks/Gaussian_Process_Latent_Variable_Model.ipynb Demo the GP library w/ MNIST GP, GPRM tfp.positive_semidefinite_kernelsNeeds TF2 update, doesn't look very hard Update
tensorflow_probability/examples/jupyter_notebooks/Gaussian_Process_Regression_In_TFP.ipynb Kind of classic GP demo GP, GPRM tfp.positive_semidefinite_kernelsNeeds TF2 update, doesn't look very hardUses AdamOptimizer, so loss needs to become a function, probably tf.function updated in cl/279842677
tensorflow_probability/examples/jupyter_notebooks/Generalized_Linear_Models.ipynb Explain GLMs, demo the tfp.glm library GLM Needs TF2 update, ed2->JD updateI don't expect a user to be able to execute the R section (suggest rpy2, https://colab.research.google.com/drive/1h5-lqnPz3WRXLXIt41N5B_aDngIIlfyL prototype, but note the r install took FOREVER for some reason)Uses tf.gradients at the bottom, consider removing that section unless we really value the didactic nature. Update
tensorflow_probability/examples/jupyter_notebooks/HLM_TFP_R_Stan.ipynb Comparing TFP (SA-EM), R, Stan for the radon dataset Hierarchical model Similar to GLMs.ipynb re: rpy2Needs TF2 migrationShould probably switch to JDAppears to autodiff through HMC? Is this still supported? Update or delete
tensorflow_probability/examples/jupyter_notebooks/Linear_Mixed_Effects_Model_Variational_Inference.ipynb VI on radon dataset dists, autodiff, trainable surrogate posterior, tfp.vi.monte_carlo_csiszar_f_divergence, tfp.vi.kl_reverse tf.make_templateuse JDTF2 migration Update or delete
tensorflow_probability/examples/jupyter_notebooks/Linear_Mixed_Effects_Models.ipynb InstEval dataset from lme4 dists, autodiff, expectation maximization Needs TF2 migration Updated in cl/282628627
tensorflow_probability/examples/jupyter_notebooks/Modeling_with_JointDistribution.ipynb JDSeq JD, HMC, batching, L-BFGS, VI, multiple datasets, MF VI, TriL VI Includes GLMM on radon, can we delete the LME notebooks?tfd.fill_triangular TODO(junpenglao): update to use NUTS, also clean up VI part. updated in cl/279789837
tensorflow_probability/examples/jupyter_notebooks/Multilevel_Modeling_Primer.ipynb Similar to HLM, LMEMVI, LMEM, GLM - nice graphics and renderings of the graphical models JD, deeper dive, and newer than prev GLMM notebooks Already TF2! Keep
tensorflow_probability/examples/jupyter_notebooks/Multiple_changepoint_detection_and_Bayesian_model_selection.ipynb Multiple changepoint inhomogenous poisson process, using HMM tfd.HMM, MAP estimation, autodiff TF2 migrationTODO(davmre): once tfp.HiddenMarkovModel implements the Viterbi algorithm to find highest-probability trajectories, update this section to use it. Update
tensorflow_probability/examples/jupyter_notebooks/Optimizers_in_TensorFlow_Probability.ipynb        
tensorflow_probability/examples/jupyter_notebooks/Probabilistic_Layers_Regression.ipynb        
tensorflow_probability/examples/jupyter_notebooks/Probabilistic_Layers_VAE.ipynb        
tensorflow_probability/examples/jupyter_notebooks/Probabilistic_PCA.ipynb       updated in cl/278789359
tensorflow_probability/examples/jupyter_notebooks/Structural_Time_Series_Modeling_Case_Studies_Atmospheric_CO2_and_Electricity_Demand.ipynb bsts, vi interface tfp.sts, tfp.vi Already TF2! Keep
tensorflow_probability/examples/jupyter_notebooks/TensorFlow_Distributions_Tutorial.ipynb        
tensorflow_probability/examples/jupyter_notebooks/TensorFlow_Probability_Case_Study_Covariance_Estimation.ipynb        
tensorflow_probability/examples/jupyter_notebooks/Understanding_TensorFlow_Distributions_Shapes.ipynb        

brianwa84 avatar Dec 10 '19 16:12 brianwa84

In addition to the example notebooks etc. mentioned above, the doc comments also have v1 code. For instance in tfp.distributions.Mixture where the main doc comment has an example using .eval() which is not needed in TF2 where eager execution is default.

mjul avatar Mar 05 '20 12:03 mjul

is this issue still open for work?

rajpratyush avatar Jul 03 '21 15:07 rajpratyush

is this issue resolved or still needs to be fixed ?

aryangupta01 avatar Oct 29 '23 20:10 aryangupta01