Scripts to batch download and convert the dataset

[RalphMao]

I wrote some simple scripts to batch-download dataset segments from google cloud and convert the original tfrecord format to KITTI format. Hope it helps.

[peisun1115]

Thank you! this is useful.

[sarimmehdi]

Hello. Where do you convert and store the calibration files in your code?

[Chr1k0]

Hey, never used gsutil before. How is the batch downloading working?

If looking here i need some project id. Whats the project id for the waymo dataset? https://colab.research.google.com/notebooks/io.ipynb#scrollTo=S7c8WYyQdh5i Or do I have to setup a project my own and somehow link the waymo data? Is it free? Just want to batch download the waymo dataset somehow....

Thanks in advance!

[peisun1115]

AFAIK, you need a Google Cloud account. It won't cost you anything.

[Chr1k0]

Yeah sure, but creating an own project (id) costs money, or not?

[peisun1115]

You can try it. I don't think it should

[RalphMao]

@Chr1k0 Creating project is not necessary. Just follow this guide: https://cloud.google.com/sdk/docs Install the sdk and run gcloud init, then you will be asked to log into your account. After that you will be free to use gsutil

[RalphMao]

@sarimmehdi The current code only downloads the image and bounding box labels. I suppose you could extract the calibration data in similar manners?

[HenryJunW]

Yeah sure, but creating an own project (id) costs money, or not?

Similar question. Does that cost money to create a new project in GCP? I would like to download the dataset using gsutil, however, it asks for a project-id, which I think I have to create a project.

[RalphMao]

@HenryJunW No, it does not cost anything in GCP.

[hongvin]

Hey, just to alert anyone who is downloading this dataset. I get charged with Download Egress cost because I am using gsutil rsync. I logined with my billing account attached.

I contacted support agent and he told me to refer here for charges, scroll up and look for "General network usage".