Performance issues in Artificial-Intelligence-Deep-Learning-Machine-Learning-Tutorials

[DLPerf]

Hello! I've found two types of performance issues in your program:

• batch() should be called before map().
• tf.Session being defined repeatedly leads to incremental overhead.

You can make your program more efficient by fixing the above two problems. Here are the tensorflow document and the Stack Overflow post to support this.

Below are detailed issues about batch() should be called before map():

• tensorflow_dl_models/official/wide_deep/wide_deep.py: dataset = dataset.batch(batch_size)(here) should be called before dataset = dataset.map(parse_csv, num_parallel_calls=5)(here).
• tensorflow_dl_models/samples/outreach/blogs/blog_estimators_dataset.py: dataset = dataset.batch(32)(here) should be called before dataset = (tf.data.TextLineDataset(file_path).skip(1).map(decode_csv))(here).
• tensorflow_dl_models/samples/outreach/blogs/blog_custom_estimators.py: .batch(32)(here) should be called before .map(decode_csv, num_parallel_calls=4)(here).
• tensorflow_dl_models/samples/core/get_started/iris_data.py: dataset = dataset.shuffle(1000).repeat().batch(batch_size)(here) should be called before dataset = dataset.map(_parse_line)(here).

Besides, you need to check the function called in map()(e.g., _parse_line called in dataset = dataset.map(_parse_line)) whether to be affected or not to make the changed code work properly. For example, if _parse_line needs data with shape (x, y, z) as its input before fix, it would require data with shape (batch_size, x, y, z).

Below are detailed issues about tf.Session being defined repeatedly:

• tensorflow_dl_models/tutorials/image/cifar10/cifar10_eval.py: with tf.Session() as sess:(here) is defined in function eval_once(here) which is repeatedly called in a loop while True:(here).
• tensorflow_dl_models/research/object_detection/eval_util.py: sess = tf.Session(master, graph=tf.get_default_graph())(here) is defined in function _run_checkpoint_once(here) which is repeatedly called in a loop while True:(here).
• tensorflow_dl_models/research/im2txt/im2txt/evaluate.py: with tf.Session() as sess:(here) is defined in function run_once(here) which is repeatedly called in a loop while True:(here).
• tensorflow_dl_models/research/capsules/experiment.py: session = tf.Session(config=tf.ConfigProto(allow_soft_placement=True))(here) is defined in function run_experiment(here) which is repeatedly called in a loop while paused < 360:(here).
• tensorflow_dl_models/research/street/python/vgsl_model.py: sess = tf.Session('')(here) is defined in a loop while True:(here).
• tensorflow_dl_models/research/skip_thoughts/skip_thoughts/track_perplexity.py: with tf.Session() as sess:(here) is defined in function run_once(here) which is repeatedly called in a loop while True:(here).
• tensorflow_dl_models/research/inception/inception/inception_eval.py: with tf.Session() as sess:(here) is defined in function _eval_once(here) which is repeatedly called in a loop while True:(here).
• tensorflow_dl_models/research/slim/datasets/download_and_convert_cifar10.py: with tf.Session('') as sess:(here) is defined in function _add_to_tfrecord(here) which is repeatedly called in a loop for i in range(_NUM_TRAIN_FILES):(here).
• deep-learning/GANs and Variational Autoencoders/BigGAN-PyTorch/scripts/tfhub/converter.py: sess = tf.Session()(here) is defined in function dump_tfhub_to_hdf5(here) and dump_tfhub_to_hdf5 is called in function convert_biggan(here) which is repeatedly called in a loop for res in RESOLUTIONS:(here).
• deep-learning/udacity-deeplearning/weight-initialization/helper.py: with tf.Session() as session:(here) is defined in function _get_loss_acc(here) which is repeatedly called in a loop for i, (weights, label) in enumerate(weight_init_list):(here).

If you define tf.Session out of the loop and pass tf.Session as a parameter to the loop, your program would be much more efficient.

Looking forward to your reply. Btw, I am very glad to create a PR to fix it if you are too busy.

[DLPerf]

Hello, I'm looking forward to your reply~