RuntimeError: numel: integer multiplication overflow

[gwilczynski95]

trafficstars

Hello. I run my custom project where I use GS rasterizer. Sometimes I encounter this problem (it happens with a random interval, usually during the second epoch, nonetheless it happens always).

Do you have any idea what is the issue here? Any tips?

23 Traceback (most recent call last):
24   File "/home/olaf/grzegos/wear_it/human_mesh_work1/torch_scripts/train_cloth_rigid_nonrigid.py", line 321, in <module>
25     main()
26   File "/home/olaf/grzegos/wear_it/human_mesh_work1/torch_scripts/train_cloth_rigid_nonrigid.py", line 313, in main
27     train(
28   File "/home/olaf/grzegos/wear_it/human_mesh_work1/torch_scripts/train_cloth_rigid_nonrigid.py", line 182, in train
29     losses, data_to_loss = iter_pass(
30   File "/home/olaf/grzegos/wear_it/human_mesh_work1/torch_scripts/train_cloth_rigid_nonrigid.py", line 59, in iter_pass
31     pred_mask = render_batch(
32   File "/home/olaf/grzegos/wear_it/human_mesh_work1/torch_scripts/own_gaussian_utils.py", line 324, in render_batch
33     rendered_image, _ = rasterizer(
34   File "/home/olaf/miniconda3/envs/splats310/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
35     return forward_call(*input, **kwargs)
36   File "/home/olaf/miniconda3/envs/splats310/lib/python3.10/site-packages/diff_gaussian_rasterization/__init__.py", line 210, in forward
37     return rasterize_gaussians(
38   File "/home/olaf/miniconda3/envs/splats310/lib/python3.10/site-packages/diff_gaussian_rasterization/__init__.py", line 32, in rasterize_gaussians
39     return _RasterizeGaussians.apply(
40   File "/home/olaf/miniconda3/envs/splats310/lib/python3.10/site-packages/diff_gaussian_rasterization/__init__.py", line 92, in forward
41     num_rendered, color, radii, geomBuffer, binningBuffer, imgBuffer = _C.rasterize_gaussians(*args)
42 RuntimeError: numel: integer multiplication overflow

For clarification. I didn't change the code for the rasterizer, I don't pass shs, just colors_precomp after sigmoid.