Scaffold-GS icon indicating copy to clipboard operation
Scaffold-GS copied to clipboard

Published 20 hours ago • verified publisher icon city-super

Training on 4K resolution

Open bernard0047 opened this issue 11 months ago • 2 comments

Hi! I'm trying to train on scenes with higher resolution, but getting worse results. What changes/optimizations do you suggest to achieve better reconstruction on 2K or 4K images?

Thanks!

bernard0047 avatar Mar 25 '24 02:03 bernard0047

Hi, could you clarify if the 'worse results' are in comparison to Scaffold's performance on lower resolution images or against 3D-GS in the same higher resolution?

inspirelt avatar Mar 25 '24 02:03 inspirelt

Hi! I'm trying to train on scenes with higher resolution, but getting worse results. What changes/optimizations do you suggest to achieve better reconstruction on 2K or 4K images?

Thanks!

I believe there is a control in CameraUtil to prevent training and evaluating resolution go beyond 1600 (in width), therefore, if you try to extend the resolution, you have to change that part.

GPU memory increase quickly using higher resolution.

yangjiheng avatar Apr 24 '24 06:04 yangjiheng

I am aware of that flag. 2K images training takes a very high memory. @inspirelt I meant the 'worse results' were as compared to Scaffold's performance on 1K resolution. I tried tweaking some hyperparameters like update_hierachy_factor, update_init_factor, update_depth, voxel_size, n_offsets, appearance_dim but got no success.

There's a similar trend in the 3D-GS pipeline too I agree.

bernard0047 avatar May 20 '24 23:05 bernard0047

Did you try to train for longer iterations? Since more details are presented in a higher resolution image, Scaffold-GS may need more growing operations.

inspirelt avatar May 21 '24 07:05 inspirelt

@bernard0047 Hi, I found the same issue in Octree-GS, which is based on Scaffold-GS. Have you find the solution or reason?

leelele-cug avatar Jun 29 '24 07:06 leelele-cug

Actually my intrinsics were wrong. With correct intrinsics I get slightly better results with higher resolution

bernard0047 avatar Jul 01 '24 21:07 bernard0047

Actually my intrinsics were wrong. With correct intrinsics I get slightly better results with higher resolution

Hi, thanks for your attention and reply.

I'm still encountering with this problem. Do you mean the intrinsics should be changed when we change the training resolution? or You just used correct intrinsics for 1.6K but use another wrong intrinsics(which suppose to be the same with 1.6K) for 4K?

leelele-cug avatar Jul 04 '24 06:07 leelele-cug

We don't need to change intrinsics if we use a larger image. Just make sure that Cx and Cy are calculated correctly (W/2, H/2) from bigger image.

My intrinsics were wrong due to some other bug and is an unrelated issue (solved)

bernard0047 avatar Jul 12 '24 18:07 bernard0047

We don't need to change intrinsics if we use a larger image. Just make sure that Cx and Cy are calculated correctly (W/2, H/2) from bigger image.

My intrinsics were wrong due to some other bug and is an unrelated issue (solved)

OK, I got it, thanks a lot.

leelele-cug avatar Jul 15 '24 07:07 leelele-cug