Memory leak when saving nifti

[oscarbates]

I have 500 CT scans which I load as 2D DICOM slices, process and save as a 3D NIFTI volume.

The NIFTI image is created using affine = np.eye(4) farray_img = nib.Nifti1Image(np_volume.astype(np.float64), affine)

and saved nib.save(farray_img, filepath)

When a file exists for the filepath, the disk memory is overwritten.

When the file does not already exist, this function appears to store the data in RAM before writing to the disk. This RAM is not released when the function call is complete, which results in a memory leak.

[effigies]

Sorry to be slow on this. Could you provide a minimal reproduction of the issue?

[oscarbates]

No problem.

I'm working with the qure.ai dataset which can be found here,

http://headctstudy.qure.ai/

I'm using python=3.6, Ubuntu18.04.3

Below is a code to replicate the bug. Sorry to quote it verbratim, I'm not sure how to attach a .py file.

import glob from natsort import natsorted import pydicom import numpy as np import copy import nibabel as nib import re

path_to_qureai_folder = "/home/Documents/"

## ~~~~~~~~~~~~ Functions ~~~~~~~~~~~~ ## def get_directories(directory, path_list): for n in range(0, len(directory)): directory[n] = directory[n] + '*/' path_list += glob.glob(directory[n]) return natsorted(path_list)

def load_dicom(filepath): n_images = len(filepath) dicoms = dict() for n in range(0, n_images): dicoms[n] = pydicom.dcmread(filepath[n]) n_images = len(dicoms) slice_pos = np.zeros(n_images) for n in range(0, n_images): slice_pos[n] = dicoms[n].SliceLocation idxs = np.argsort(slice_pos) n = -1 image_data_temp = copy.deepcopy(dicoms) for id in idxs: n = n + 1 dicoms[n] = image_data_temp[id] n_images = len(dicoms) pixel_data = np.zeros((dicoms[0].Rows, dicoms[0].Columns, n_images), dtype=float) for n in range(0, n_images): pixel_data[:, :, n] = dicoms[n].pixel_array

pixel_data = np.array(pixel_data, np.float32) return pixel_data, dicoms

## ~~~~~~~~~~~~ Code ~~~~~~~~~~~~ ##

directory = [path_to_qureai_folder+"qureai/"] path_list = list() path_list = get_directories(directory, path_list) path_list = get_directories(directory, path_list) path_list = get_directories(directory, path_list)

for volume_id in range(0, len(path_list)): folderpath = [path_list[volume_id]] dcm_list = list() for n in range(0, len(folderpath)): dcm_list += glob.glob(folderpath[n] + '*.dcm') natsorted(dcm_list)

if len(dcm_list) != 0: volume, header = load_dicom(dcm_list)

affine = np.eye(4) farray_img = nib.Nifti1Image(volume.astype(np.float64), affine)

pattern = re.compile(r'\bCQ') # Find folder-name filename_matches = pattern.finditer(path_list[volume_id]) for match in filename_matches: filename_pos0 = match.span()[0]

pattern = re.compile(r'[0123456789]/') # Find folder-name filename_matches = pattern.search(path_list[volume_id]) filename_pos1 = filename_matches.span()[1] - 1

filename = path_list[volume_id][filename_pos0:filename_pos1] filepath = path_to_qureai_folder + 'qureai_nii2/' \ + str(volume_id) + '_' + filename + '.nii'

#### Memory leak in save? Try make file first, then save, then close file #### # f=open(filepath,"w+") nib.save(farray_img, filepath) # f.close()

[effigies]

You can paste files in a gist.