Possible memory leak

[eirikbrandsaas]

On my system (Linux - Pop!_OS 20.04) running makePSID("user_input.json") works fine, but it uses a lot of memory (~10gb). However, for some reason the memory is never released after the function is done running. In fact at one point it actually crashed my computer

The memory usage increase is when the package reads in the data (unzip_data.jl)

datas =  SortedDict(year => readPSID(filename) for (year, filename) in zip(years, filenames))

• After running makePSID() I use 90% of my memory, when I had used ~30% before.
• If I run the command a second time, sometimes it works, sometimes not. (i.e. sometimes I can finish building the datafile without Julia crashing and my laptop hanging).

I dont understand why the memory used in datas isnt released (or whatever the proper terminology is) after the function finishes.

What I have tried to do:

1. Added GC.gc() after some of the function calls in makePSID. Maybe this helped a little? It seemed to reduce my memeory use to ~80%. So clearly something else is still going on.

    famdatas, inddata = PSID.unzip_data()
    GC.gc()
    println("Constructing data")
    PSID.construct_alldata(famdatas, inddata, codemissings = codemissings)
    GC.gc()