deepmind-research icon indicating copy to clipboard operation
deepmind-research copied to clipboard

How to generate train.tfrecord?

Open cwl1999 opened this issue 3 years ago • 8 comments

Dear author: Hello! I am a graduate student from Wuhan Textile University in China. I am studying your project about learning recently_ to_ simulate. I'm very interested in this. Can you provide the generated data train.tfrecord Source dataset file for? My email address is: [email protected]. Thank you very much for your reading and look forward to your reply.

cwl1999 avatar Apr 23 '21 03:04 cwl1999

Thank you for your message?

Can you provide the generated data train.tfrecord Source dataset file for?

Could you clarify the question/which datasets you are referring to? All datasets should be downloadable by name using the download script, as per the instructions in the README:

mkdir -p /tmp/datasets
bash ./learning_to_simulate/download_dataset.sh WaterRamps /tmp/datasets

Hope this helps.

alvarosg avatar Aug 18 '21 11:08 alvarosg

Thank you very much for your answer! Because I want to try to make a cloth dataset for simulation experiments, but it is difficult for me to obtain the data content of tf.records file. When I forcibly open it, I can only see the garbled code. I'm confused now. Please give me some advice.

cwl1999 avatar Aug 20 '21 02:08 cwl1999

Thanks for your reply. If you want human readable access, I would recommend using python to read the dataset and iterate through the examples. You may use TF2 to do this:

    import functools
    import tensorflow as tf
    from learning_to_simulate import reading_utils

    ds = tf.data.TFRecordDataset([os.path.join(data_path, f'{split}.tfrecord')])
    ds = ds.map(functools.partial(
        reading_utils.parse_serialized_simulation_example, metadata=metadata))

    for element in ds.as_numpy_iterator():
      print(element)

alvarosg avatar Aug 23 '21 09:08 alvarosg

    for element in ds.as_numpy_iterator():
    for element in ds:

aakash30jan avatar Dec 16 '21 18:12 aakash30jan

I have a simular question. I want to creat a test dataset with ramps set by myself. What should I do? How can I creater such a test.tfrecord file?

Social-Mean avatar Dec 18 '22 02:12 Social-Mean

Thanks for your reply. If you want human readable access, I would recommend using python to read the dataset and iterate through the examples. You may use TF2 to do this:

    import functools
    import tensorflow as tf
    from learning_to_simulate import reading_utils

    ds = tf.data.TFRecordDataset([os.path.join(data_path, f'{split}.tfrecord')])
    ds = ds.map(functools.partial(
        reading_utils.parse_serialized_simulation_example, metadata=metadata))

    for element in ds.as_numpy_iterator():
      print(element)

What is metadata? Do I need to change reading_utils according to different dataset?

leixinma717 avatar Oct 10 '23 19:10 leixinma717