neuralsim
neuralsim copied to clipboard
running with custom dataset
I'm trying to run StreetSurf with my own dataset (only posed camera data) and have a few questions and I'm wondering whether you can help with that.
I'm doing these steps:
- Convert my data to the the data format described in the doc. I populated the camera data in the 'observers' fields ('hw', 'intr', 'c2w' etc).
- Run the depth/normal and mask prior extraction following the doc.
- Run training with the config withmask_nolidar.230814.yaml. I updated
camera_list
, theroot
in thedataset_cfg
andscenarios
inscene bank_cfg
accordingly and others unchanged.
I have two questions:
- In the doc, the 'ego_car' is commented as optional, but when I run
train.py
it complains about missing keyscenario['observers']['ego_car']['data']['v2w']
. Since I already provided 'c2w' in the camera data I'm wondering why vehicle pose is still required? My cameras are not synced so what vehicle poses should I use? - The ego car hood takes a large part in the front camera image, is there a way to consume the ego car mask? I'm cropping the images for now but some valid pixels are lost in this process.
Also one minor thing: The doc reads scenario['metas']['num_frames']
but the code seems to require scenario['metas']['n_frames']
.