research-contributions icon indicating copy to clipboard operation
research-contributions copied to clipboard

Validation list can case division by zero error if empty

Open holgerroth opened this issue 1 year ago • 0 comments

https://github.com/Project-MONAI/research-contributions/blob/171aeaa345f56d588939923c8bbd0fa7c971df88/auto3dseg/algorithm_templates/dints/scripts/search.py#L152

A collaborator ran into this issue only after one full training (several hundred epochs). The length of validation set should be checked earlier to surface this misconfiguration error before training happens.

Traceback (most recent call last):
  File "/dataT0/Free/mohno/autoseg/autoseg1219/dints_4/scripts/search.py", line 652, in <module>
    fire.Fire()
  File "/dataT0/Free/mohno/.local/lib/python3.7/site-packages/fire/core.py", line 141, in Fire
    component_trace = _Fire(component, args, parsed_flag_args, context, name)
  File "/dataT0/Free/mohno/.local/lib/python3.7/site-packages/fire/core.py", line 471, in _Fire
    target=component.__name__)
  File "/dataT0/Free/mohno/.local/lib/python3.7/site-packages/fire/core.py", line 681, in _CallAndUpdateTrace
    component = fn(*varargs, **kwargs)
  File "/dataT0/Free/mohno/autoseg/autoseg1219/dints_4/scripts/search.py", line 143, in run
    val_files = val_files * math.ceil(float(world_size) / float(len(val_files)))
ZeroDivisionError: float division by zero

holgerroth avatar Jan 10 '23 13:01 holgerroth