boltz predict should fail (return nonzero) when it runs out of memory

[timodonnell]

It would be helpful if boltz predict would return a nonzero status when it fails with errors like:

LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
Predicting DataLoader 0:   0%|          | 0/1 [00:00<?, ?it/s]| WARNING: ran out of memory, skipping batch
Predicting DataLoader 0: 100%|██████████| 1/1 [07:08<00:00,  0.00it/s]Number of failed examples: 1
Predicting DataLoader 0: 100%|██████████| 1/1 [07:08<00:00,  0.00it/s]

as it stands now the command succeeds but it gives no predicted structures.

The nonzero status would be useful so errors get propagated upward in scripts and wrappers.

[gcorso]

That makes sense, what would you think the behavior should be in the case of a multi-complex prediction run in which some but not all the complexes fail?

[timodonnell]

My vote would be that the command should fail if any predictions fail, even if others succeed.