viser
viser copied to clipboard
gengshan-y
can not train on davis car turn
why train on a subset of video(id 22-id42 in the breakdance-flare)? why not train with the whole video
How is the data pre-processed? The pre-processing script hasn't been added yet.
why train on a subset of video(id 22-id42 in the breakdance-flare)?
We start from random root poses (equivalent to camera pose). Training on all videos does not always produce correct root pose of the dancer, possibly due to limited batch size.
In this work, moving camera is treated as static camera + moving root body.
batchsize 4 only cost 4Ggpu memory. you can improve the batchsize. Are you saying that the larger the batchsize, the better performance?
I used your preprocess code (https://github.com/gengshan-y/viser-release/blob/main/preprocess/README.md) but got stuck (pdb) where you break. Commenting out can be trained, but it seems that the program has bugs. you can try it with 'car-turn'
Isn't this place data preprocessing code? (https://github.com/gengshan-y/viser-release/blob/main/preprocess/README.md)
Can you point me where the break happens?
Unfortunately, I have limited capacity do further experiments. In my experience, larger batch size stabilizes training and improves performance. But note that solving dynamic shapes with large deformation from 2D is very under-constrained. I cannot guarantee it works even 2x batch size is used.
You are welcome to test it and let me know if that works.
Re
Isn't this place data preprocessing code? (https://github.com/gengshan-y/viser-release/blob/main/preprocess/README.md)
I need to make it clear that the codebase's preprocessing code is complete
I remembered it wrong and the pre-processing code was tested. I would suggest do the following.
- check whether flow is correctly computed in the FlowFW/Full-Resolution/$sequence-name/ folder
- check whether the code loads "black" flow images for breakdance sequence.
- find the difference of data format between your sequence and breakdance sequence