struct.error: unpack requires a buffer of 4 bytes

[guyko81]

I wanted to use the simple bicycle example under Windows 11

accelerate launch train.py --gin_configs=configs/360.gin --gin_bindings="Config.data_dir = 'data/bicycle'" --gin_bindings="Config.exp_name = 'bicycle'" --gin_bindings="Config.factor = 4"

and got this error

[SuLvXiangXin]

@guyko81 Please type all of your error message. I guess this is because your bicycle data is broken, please check it.

[guyko81]

I found a solution by changing 8 to 4 in the read and read the remaining 4 bytes after, so the following codes still work

f.seek(0)  # reset the file cursor to the beginning of the file
num_cameras = struct.unpack('L', f.read(4))[0] # instead of 8
f.read(4)  # skip the 4 bytes after
[...]
camera_id, camera_type, w, h = struct.unpack('IiLL', f.read(16)) # instead of 24

...
_load_images_bin:
num_images = struct.unpack('L', f.read(4)[0] # instead of 8
f.read(4)  # skip

I'm happy to hear what could've been the issue, but it worked.

[TeamMasse]

I had a similar Problem using this with Windows 10. My error message was:

(zipnerf) PS C:\Users\Admin\Documents\zipnerf-pytorch> accelerate launch train.py --gin_configs=configs/360.gin --gin_bindings="Config.data_dir = '${DATA_DIR}'" --gin_bindings="Config.exp_name = '${EXP_NAME}'" --gin_bindings="Config.factor = 4"
2023-07-01 11:45:19: Config(dataset_loader='llff', batching='all_images', batch_size=65536, patch_size=1, factor=4, multiscale=False, multiscale_levels=4, forward_facing=False, render_path=False, llffhold=8, llff_use_all_images_for_training=False, llff_use_all_images_for_testing=False, use_tiffs=False, compute_disp_metrics=False, compute_normal_metrics=False, disable_multiscale_loss=False, randomized=True, near=0.2, far=1000000.0, exp_name='360_v2/bicycle', data_dir='data/360_v2/bicycle', vocab_tree_path=None, render_chunk_size=65536, num_showcase_images=5, deterministic_showcase=True, vis_num_rays=16, vis_decimate=0, max_steps=25000, early_exit_steps=None, checkpoint_every=5000, resume_from_checkpoint=True, checkpoints_total_limit=1, gradient_scaling=False, print_every=100, train_render_every=500, data_loss_type='charb', charb_padding=0.001, data_loss_mult=1.0, data_coarse_loss_mult=0.0, interlevel_loss_mult=0.0, anti_interlevel_loss_mult=0.01, orientation_loss_mult=0.0, orientation_coarse_loss_mult=0.0, orientation_loss_target='normals_pred', predicted_normal_loss_mult=0.0, predicted_normal_coarse_loss_mult=0.0, hash_decay_mults=0.1, lr_init=0.01, lr_final=0.001, lr_delay_steps=5000, lr_delay_mult=1e-08, adam_beta1=0.9, adam_beta2=0.99, adam_eps=1e-15, grad_max_norm=0.0, grad_max_val=0.0, distortion_loss_mult=0.005, opacity_loss_mult=0.0, eval_only_once=True, eval_save_output=True, eval_save_ray_data=False, eval_render_interval=1, eval_dataset_limit=2147483647, eval_quantize_metrics=True, eval_crop_borders=0, render_video_fps=60, render_video_crf=18, render_path_frames=120, z_variation=0.0, z_phase=0.0, render_dist_percentile=0.5, render_dist_curve_fn=<ufunc 'log'>, render_path_file=None, render_resolution=None, render_focal=None, render_camtype=None, render_spherical=False, render_save_async=True, render_spline_keyframes=None, render_spline_n_interp=30, render_spline_degree=5, render_spline_smoothness=0.03, render_spline_interpolate_exposure=False, rawnerf_mode=False, exposure_percentile=97.0, num_border_pixels_to_mask=0, apply_bayer_mask=False, autoexpose_renders=False, eval_raw_affine_cc=False, zero_glo=False, valid_weight_thresh=0.05, isosurface_threshold=20, mesh_voxels=134217728, visibility_resolution=512, mesh_radius=1.0, mesh_max_radius=10.0, std_value=0.0, compute_visibility=False, extract_visibility=True, decimate_target=-1, vertex_color=True, vertex_projection=True, tsdf_radius=2.0, tsdf_resolution=512, truncation_margin=5.0, tsdf_max_radius=10.0)
2023-07-01 11:45:19: Distributed environment: NO
Num processes: 1
Process index: 0
Local process index: 0
Device: cuda

Mixed precision type: fp16

Warning: image_path not found for reconstruction
2023-07-01 11:45:19: Error!
Traceback (most recent call last):
  File "C:\Users\Admin\Documents\zipnerf-pytorch\train.py", line 387, in <module>
    app.run(main)
  File "C:\Users\Admin\.conda\envs\zipnerf\lib\site-packages\absl\app.py", line 308, in run
    _run_main(main, args)
  File "C:\Users\Admin\.conda\envs\zipnerf\lib\site-packages\absl\app.py", line 254, in _run_main
    sys.exit(main(argv))
  File "C:\Users\Admin\Documents\zipnerf-pytorch\train.py", line 73, in main
    dataset = datasets.load_dataset('train', config.data_dir, config)
  File "C:\Users\Admin\Documents\zipnerf-pytorch\internal\datasets.py", line 39, in load_dataset
    return dataset_dict[config.dataset_loader](split, train_dir, config)
  File "C:\Users\Admin\Documents\zipnerf-pytorch\internal\datasets.py", line 278, in __init__
    self._load_renderings(config)
  File "C:\Users\Admin\Documents\zipnerf-pytorch\internal\datasets.py", line 543, in _load_renderings
    pose_data = NeRFSceneManager(colmap_dir).process()
  File "C:\Users\Admin\Documents\zipnerf-pytorch\internal\datasets.py", line 61, in process
    self.load_cameras()
  File "C:\Users\Admin\Documents\zipnerf-pytorch\internal/pycolmap/pycolmap\scene_manager.py", line 90, in load_cameras
    self._load_cameras_bin(input_file)
  File "C:\Users\Admin\Documents\zipnerf-pytorch\internal/pycolmap/pycolmap\scene_manager.py", line 102, in _load_cameras_bin
    num_cameras = struct.unpack('L', f.read(8))[0]
struct.error: unpack requires a buffer of 4 bytes
Traceback (most recent call last):
  File "C:\Users\Admin\.conda\envs\zipnerf\lib\runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Users\Admin\.conda\envs\zipnerf\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\Users\Admin\.conda\envs\zipnerf\Scripts\accelerate.exe\__main__.py", line 7, in <module>
  File "C:\Users\Admin\.conda\envs\zipnerf\lib\site-packages\accelerate\commands\accelerate_cli.py", line 45, in main
    args.func(args)
  File "C:\Users\Admin\.conda\envs\zipnerf\lib\site-packages\accelerate\commands\launch.py", line 941, in launch_command
    simple_launcher(args)
  File "C:\Users\Admin\.conda\envs\zipnerf\lib\site-packages\accelerate\commands\launch.py", line 603, in simple_launcher
    raise subprocess.CalledProcessError(returncode=process.returncode, cmd=cmd)
subprocess.CalledProcessError: Command '['C:\\Users\\Admin\\.conda\\envs\\zipnerf\\python.exe', 'train.py', '--gin_configs=configs/360.gin', "--gin_bindings=Config.data_dir = 'data/360_v2/bicycle'", "--gin_bindings=Config.exp_name = '360_v2/bicycle'", '--gin_bindings=Config.factor = 4']' returned non-zero exit status 1.

I could fix it with the code provided by @guyko81 but he forgot one bracket it should actually be:

num_images = struct.unpack('L', f.read(4)[0] # instead of 8

[WPCJATH]

My solution is to replace all 'L' with 'Q' instead of changing the number of bytes to read: struct.unpack('Q', f.read(8)[0] And this works very well. Thanks to ChatGPT.

[WPCJATH]

My solution is to replace all 'L' with 'Q' instead of changing the number of bytes to read: struct.unpack('Q', f.read(8)[0] And this works very well. Thanks to ChatGPT.

'Q' means unsigned long long integer.