FESTIM icon indicating copy to clipboard operation
FESTIM copied to clipboard

TXT export: store data in memory and write only at last timestep

Open RemDelaporteMathurin opened this issue 1 year ago • 4 comments

@jhdark suggested that instead of writing the TXT at each timestep, we could store the data in memory and write everything at the last time step.

We could have a user argument to choose the behaviour.

@KulaginVladimir what do you think?

RemDelaporteMathurin avatar Aug 02 '24 11:08 RemDelaporteMathurin

We could have a user argument to choose the behaviour.

Makes sense.

KulaginVladimir avatar Aug 02 '24 12:08 KulaginVladimir

We could have a user argument to choose the behaviour.

Do we need to write data at each timestep?

In the current implementation, TXTExport writes data -> reads old data and appends new -> writes data, and so on. The only benefit I see is that at some timestep we can run out of memory and get an error, but the already written data will remain (?). However, we nonetherless read the whole dataset at each call of TXTExport.write()

KulaginVladimir avatar Sep 08 '24 22:09 KulaginVladimir

We are working with numpy arrays so it's unlikely that we will run out of memory.

Plus I don't see how not writing the data at each timestep would solve this? Maybe there's something I don't get here

RemDelaporteMathurin avatar Sep 09 '24 12:09 RemDelaporteMathurin

We are working with numpy arrays so it's unlikely that we will run out of memory.

I guess, it depends on the export parameters (number of export times and number of vertices).

Plus I don't see how not writing the data at each timestep would solve this? Maybe there's something I don't get here

It won't. I meant that there is no much difference. Maybe, just make TXTExport to write data at last export time?

KulaginVladimir avatar Sep 09 '24 12:09 KulaginVladimir