gen_parallel_benchmark hangs.

[zbigniew-polito]

error_report.txt I ran onto some problems when executing gen_parallel_benchmark, it looks like hangs

[eternalNight]

Not sure if the model is referenced in any global variable that defers its deletion.

Also, I think we should consider cleaning up duplicated definitions of print_rank_0 and replacing them with calls to the global logger (after extending it with those multi-gpu debugging facilities).