No data in the folder

[imhuaweizhu]

I followed the link provided by Waymo, and downloaded the waymo_open_dataset_motion_v_1_2_0, it only has 6 Bytes, basically empty.

Am I missing something?

[scott-ettinger]

Sorry for the trouble on this. There are two directories in the link you shared. One is named "uncompressed", the other is named "uncompressed_$folder$" and is an artifact of the GCP file system. The "uncompressed" directory contains the dataset while "uncompressed_$folder$" is only 6 bytes.

Could you check to see if you tried to download the correct folder?

I will remove the extra "uncompressed_$folder$" directory.

[imhuaweizhu]

Thanks for the reply.

I tried first one too, The "uncompressed" directory. I tried to download the data with this command gsutil -m cp -r "gs://waymo_open_dataset_motion_v_1_2_0/uncompressed" . in my terminal, then I got the following error.

Unknown option: m
No command was given.

Choose one of -b, -d, -e, or -r to do something.
Try `/usr/bin/gsutil --help' for more information.

[scott-ettinger]

I tried the following command on my system and it works: gsutil -m cp -r "gs://waymo_open_dataset_motion_v_1_2_0/uncompressed" .

It looks like your error is around the -m flag not being supported. Could you check is to see which version of gsutil you are using by running:

gsutil version 

Could you also try running

gsutil help options

To see if it lists the -m option?

Once this works, you may want to download only the format you plan to use. If you plan to use scenario protos, use: gsutil -m cp -r "gs://waymo_open_dataset_motion_v_1_2_0/uncompressed/scenario" .

Note that the optional lidar data is in a separate directory.

The only other thing I would try is removing the quotation marks.
Please let us know if you are still having trouble.

[imhuaweizhu]

There is no -m option

[imhuaweizhu]

@scott-ettinger Could it be the wrong version of gsutil is installed?

[scott-ettinger]

Yes, it looks like you may have installed a different utility. You will need to install the google cloud CLI which includes the gsutil command line utility. You can find instructions here: https://cloud.google.com/sdk/docs/install

[shreyas1230]

I have a similar question but regarding the waymo_open_dataset_v_2_0_0 folder. Most of the camera_segmentation parquet files in the validation set and a few in the training set have size 10KB and when I try to open them and view the contents as a Dask dataframe, the number of rows in the dataframe is 0. After going through all the segmentation files in the validation directory, it seems like there are only around 20 nonempty files.

In contrast, all the camera images in the validation set are present. Looking through the images for one of the parquet files (validation/camera_image/10203656353524179475_7625_000_7645_000.parquet), it seems like the corresponding segmentation labels should be present since there are quite a few cars in view. Is the absence of segmentation label data for most of these sequences expected?

[trand2k]

I use this command for download waymo data

gcloud storage cp -r gs://waymo_open_dataset_motion_v_1_2_0/uncompressed/scenario .

hope this can help

[thuongtrannc]

I use this command for download waymo data

gcloud storage cp -r gs://waymo_open_dataset_motion_v_1_2_0/uncompressed/scenario .

hope this can help

It worked for me. Thanks alot @trand2k