Batch norm bug - BaselineMethod, Multi-Task Synbols

[lebrice]

sequoia_sweep --setting multi_task --max_runs 100 --dataset synbols --nb_tasks 12 \
 --method baseline --encoder resnet18 --database_path orion/multi_task_synbols_baseline.pkl \
 --project synbols_debug --monitor_training_performance True

error_multitask.pdf

[lebrice]

Just for more context: This appears to be caused by the input image in synbols being only 32x32, which, when using a resnet18, produces representations with only one value per channel, which raises an error when trying to use BatchNorm layer on them.

We could probably resize the image to make it bigger, but that would probably be very inefficient... I'll need to think a bit more about this.