dataset.make_one_shot_iterator() raises AttributeError: 'MapDataset' object has no attribute 'make_one_shot_iterator'

[skeller88]

make_one_shot_iterator has been deprecated in tensorflow V2, but petastorm documentation still recommends using it. That results in an exception:

'MapDataset' object has no attribute 'make_one_shot_iterator'

The new approach seems to be:

with make_reader(output_url, num_epochs=num_epochs) as reader:
    dataset = make_petastorm_dataset(reader)
    for data in dataset:
        print(data.id)

So the documentation should reflect that.

Similarly, tf_tensors raises an exception:

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-33-a5e1f608f655> in <module>
      3 
      4 with make_reader(output_url, num_epochs=2) as reader:
----> 5     dataset = tf_tensors(reader)
      6     for data in dataset:
      7         print(data)

/opt/conda/lib/python3.7/site-packages/petastorm/tf_utils.py in tf_tensors(reader, shuffling_queue_capacity, min_after_dequeue)
    334         result = _tf_tensors_ngram(reader, shuffling_queue_capacity, min_after_dequeue)
    335     else:
--> 336         result = _tf_tensors_nonngram(reader, shuffling_queue_capacity, min_after_dequeue)
    337 
    338     return result

/opt/conda/lib/python3.7/site-packages/petastorm/tf_utils.py in _tf_tensors_nonngram(reader, shuffling_queue_capacity, min_after_dequeue)
    234     # fields_as_list is a list with tensors matching the order of the values in the schema. named-tuple semantics is
    235     # not preserved across tf.py_func call boundary.
--> 236     fields_as_list = tf.py_func(dequeue_sample_impl, [tf.constant(1)], _schema_to_tf_dtypes(reader.schema))
    237 
    238     if shuffling_queue_capacity > 0:

AttributeError: module 'tensorflow' has no attribute 'py_func'

In tf 2, py_func has been replaced by py_function.

It would be nice if tf_utils had methods that were compatible with tensorflow 2, or at least if that were called out in the documentation.

[selitvin]

Yep. It's probably time to make sure the code is compatible with TF2. It will take some time I guess, but thanks for bringing it to my attention. Will start looking at it.