Bug: Pickling error when processing large files.

[adeane-ga]

Description

When processing with large files (usually because of higher resolution), the below error message is raised and PyRate fails. It is to do with the use of the pickle package to transfer files around mid process.

Error Message

Traceback (most recent call last): File "/home/547/ad6200/PyRateVenv/bin/pyrate", line 11, in load_entry_point('Py-Rate==0.5.0', 'console_scripts', 'pyrate')() File "/home/547/ad6200/PyRateVenv/lib/python3.7/site-packages/pyrate/main.py", line 116, in main correct.main(config) File "/home/547/ad6200/PyRateVenv/lib/python3.7/site-packages/pyrate/correct.py", line 142, in main return correct_ifgs(config) File "/home/547/ad6200/PyRateVenv/lib/python3.7/site-packages/pyrate/correct.py", line 231, in correct_ifgs correct_steps[step](params, config) File "/home/547/ad6200/PyRateVenv/lib/python3.7/site-packages/pyrate/correct.py", line 167, in phase_closure_wrapper rets = iterative_closure_check(config) File "/home/547/ad6200/PyRateVenv/lib/python3.7/site-packages/pyrate/core/phase_closure/closure_check.py", line 129, in iterative_closure_check rets = __wrap_closure_check(config) File "/home/547/ad6200/PyRateVenv/lib/python3.7/site-packages/pyrate/core/phase_closure/closure_check.py", line 206, in __wrap_closure_check closure, ifgs_breach_count, num_occurences_each_ifg = sum_phase_closures(ifgs_with_loops, retained_loops, params) File "/home/547/ad6200/PyRateVenv/lib/python3.7/site-packages/pyrate/core/phase_closure/sum_closure.py", line 79, in sum_phase_closures edge_to_indexed_ifgs = __create_ifg_edge_dict(ifg_files, params) File "/home/547/ad6200/PyRateVenv/lib/python3.7/site-packages/pyrate/core/phase_closure/sum_closure.py", line 60, in __create_ifg_edge_dict ret_combined = join_dicts(mpiops.comm.allgather(ret_combined)) File "mpi4py/MPI/Comm.pyx", line 1272, in mpi4py.MPI.Comm.allgather File "mpi4py/MPI/msgpickle.pxi", line 770, in mpi4py.MPI.PyMPI_allgather File "mpi4py/MPI/msgpickle.pxi", line 775, in mpi4py.MPI.PyMPI_allgather File "mpi4py/MPI/msgpickle.pxi", line 148, in mpi4py.MPI.Pickle.allocv File "mpi4py/MPI/msgpickle.pxi", line 139, in mpi4py.MPI.Pickle.alloc SystemError: Negative size passed to PyBytes_FromStringAndSize

Task

1. Identify where this is happening in the process.
2. Identify how it can be solved.

Replication

To replicate error, process any data with multilook factor of 1 (no multilooking) - full resolution. Even heavily cropped areas can still present this issue if no multilooking is done.