training car task crashes on missing kitti_train_object_cls.tfrecord file

[larawilson88]

Hi, I'm trying to run the car task for kitti data and training fails on a "kitti_train_object_cls.tfrecord" file ( i assume it's for augmentation). I have created the train, val and test tfrecords using kitti_exporter tool, but could not find the code to create this missing one. this one seems to have 19700 items which i think are all the objects bounding boxes. could you please provide this file or tool to generate this. thanks Lara

[drpngx]

Do you have the log file?

On Sat, Nov 23, 2019 at 5:46 PM larawilson88 [email protected] wrote:

Hi, I'm trying to run the car task for kitti data and training fails on a "kitti_train_object_cls.tfrecord" file ( i assume it's for augmentation). I have created the train, val and test tfrecords using kitti_exporter tool, but could not find the code to create this missing one. this one seems to have 19700 items which i think are all the objects bounding boxes. could you please provide this file or tool to generate this. thanks Lara

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/tensorflow/lingvo/issues/173?email_source=notifications&email_token=AE75E3JPTAQVYVRTMGW5IFTQVHMGXA5CNFSM4JQ4Z3Y2YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4H3TCTCA, or unsubscribe https://github.com/notifications/unsubscribe-auth/AE75E3NP5WU7SXNA25YH3RDQVHMGXANCNFSM4JQ4Z3YQ .

[larawilson88]

i can run the code again if you need the log, but the error was very simple: no files with format 'kitti_train_object_cls.tfrecord-*-of-00100' were found (it's referenced in (https://github.com/tensorflow/lingvo/blob/master/lingvo/tasks/car/params/kitti.py#L312)

i used to get the same error for the 'kitti_object_3dop_train.tfrecord-*-of-00100' files (referenced in the same file https://github.com/tensorflow/lingvo/blob/master/lingvo/tasks/car/params/kitti.py#L48) but i managed to create those using the kitti_exporter.py tool and that issue resolved. however, I could not find the code to create the missing 'objects' tfrecords. @vrv

[drpngx]

Did you run the kitti exporter to create the tfrecord files?

On Sun, Nov 24, 2019 at 4:59 PM larawilson88 [email protected] wrote:

i can run the code again if you need the log, but the error was very simple: no files with format 'kitti_train_object_cls.tfrecord-*-of-00100' were found (it's referenced in ( https://github.com/tensorflow/lingvo/blob/master/lingvo/tasks/car/params/kitti.py line 312)

i used to get the same error for the 'kitti_object_3dop_train.tfrecord-*-of-00100' files (referenced in the same file https://github.com/tensorflow/lingvo/blob/master/lingvo/tasks/car/params/kitti.py line 48) but i managed to create those using the kitti_exporter.py tool and that issue resolved. however, I could not find the code to create the missing 'objects' tfrecords.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/tensorflow/lingvo/issues/173?email_source=notifications&email_token=AE75E3LQEZWKDJEUFPY47WLQVMPNLA5CNFSM4JQ4Z3Y2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEFA2CWI#issuecomment-557949273, or unsubscribe https://github.com/notifications/unsubscribe-auth/AE75E3LTGCVAGQMES7TYOMTQVMPNLANCNFSM4JQ4Z3YQ .

[larawilson88]

yes, as mentioned in my previous comment i ran kitti_exporter.py. kitti exporter can produce 3 file sets, the missing one is not among those:

1. 'kitti_object_3dop_train.tfrecord-*-of-00100' (used in line 48 of kitti.py mentioned in last comment) size= 3712

2. 'kitti_object_3dop_val.tfrecord-*-of-00100' (used in 56 line of kitti.py) size=3769

3. 'kitti_object_test.tfrecord-*-of-00100' (used in line 64 of kitti.py) size = 7518

the file that is missing 'kitti_train_object_cls.tfrecord-*-of-00100' is not any of these and based on line 317 has a size of 19700 and a different format than these 3. it looks like another code should generate these.

my question is how should i generate 'kitti_train_object_cls.tfrecord-*-of-00100' files since kitti exporter does not do that.

[vrv]

@larawilson88 it looks like we forgot to opensource this file. We'll try to publish it tomorrow and you can take a look at how we generate it - it currently uses Apache Beam to process the records.

We appreciate you trying to use our code, and sorry for the trouble!

[vrv]

@larawilson88 let us know if the file we pushed two days ago helps in generating the data.

[yinjunbo]

@larawilson88 Hi, did you successfully train on the KITTI with the GT data augmentation?