neuralsim icon indicating copy to clipboard operation
neuralsim copied to clipboard

running with custom dataset

Open xslittlegrass opened this issue 1 year ago • 1 comments

I'm trying to run StreetSurf with my own dataset (only posed camera data) and have a few questions and I'm wondering whether you can help with that.

I'm doing these steps:

  1. Convert my data to the the data format described in the doc. I populated the camera data in the 'observers' fields ('hw', 'intr', 'c2w' etc).
  2. Run the depth/normal and mask prior extraction following the doc.
  3. Run training with the config withmask_nolidar.230814.yaml. I updated camera_list, the root in the dataset_cfg and scenarios in scene bank_cfg accordingly and others unchanged.

I have two questions:

  1. In the doc, the 'ego_car' is commented as optional, but when I run train.py it complains about missing key scenario['observers']['ego_car']['data']['v2w']. Since I already provided 'c2w' in the camera data I'm wondering why vehicle pose is still required? My cameras are not synced so what vehicle poses should I use?
  2. The ego car hood takes a large part in the front camera image, is there a way to consume the ego car mask? I'm cropping the images for now but some valid pixels are lost in this process.

Also one minor thing: The doc reads scenario['metas']['num_frames'] but the code seems to require scenario['metas']['n_frames'].

xslittlegrass avatar Sep 13 '23 08:09 xslittlegrass