Muhammad Bilal

i think problem arises due to '''''''''''''''''''''''''' if 'params.' in path: path = path.replace('params.', '') '''''''''''''''''''''''''''''''' in _ModelParamsClassKey but when we read a specific class its not removing this string....

Thanks! but when i use TPU type v3-8 for training instead of v3-32(which is really expensive and only available for evaluation). I get this error: 2020-04-12T21:48:19.580902Z params.train.max_steps: 742317, enqueue_max_steps: -1...

is it possible to train waymo model on cloud gpu? If yes, which one should be the best to use?

I have tried again i am getting same error. I have checked that all the machines are running. Although I have noted that in gke_launch.py we are using the following...

I run it on cloud gpu and i got read error on waymo repository although I can open the link in the browser and can download the files? what should...

I am still having the same error.

its the same as I can open the repository in web browser. Also i can use gsutil cp on it.

they have given access to user accounts but cloud vm use service accounts (*********[email protected]) to communicate. Is there any workaround to override this?

it's really expensive to copy from one repository to another.

@vrv can you tell how much time it will take to train waymo vehicle model on cloud TPU?