compute_laplacian_matrices saving_address

[YasAsgari]

Hi, I am now running this step. By the code:

fs.compute_laplacian_matrices()

The procedure is fast enough and has a very good log. But the point is that I cannot save anything in any address to reuse it next time. Is there any possibility to save the object somewhere?

I checked the documentation def compute_laplacian_matrices(self, **kwargs): self._temporal_network.compute_laplacian_matrices(**kwargs) return self

From my understanding this is only a wrapper.

So if I want to save it, I have to use:

'def save(self, filename, matrices_list=None, attributes_list=None):`

And then it is not possible to reload it.

So for me it is quite not easy to understand how to save everything for further usage.

Suggestions for now:

Maybe add a save option to the flow stability class. (I am not sure)

[alexbovet]

agreed. it should be possible to save/load the current state of a FlowStability instance.

@j-i-l , can you suggest the best way to implement this? we can reuse the methods from ContTempNetwork.

[j-i-l]

FlowStability instances are pickle-able so they can be stored with the pickle module.

If we wanted a "save" way to store and shre FlowStability instances then we might want to consider json. However, serializing all attributes, in particular the attributes of the related ContTempNetwork instance might be a bit of work (we would need to check).

Should we implement a save and a load method as interface for pickle.dump and pickle.load ?

[j-i-l]

Or would we want to export only the laplacian matices and need a dedicated method for this?

[alexbovet]

In the end, I was using that to save the laplacians and network. https://github.com/alexbovet/flow_stability/blob/434caf46c06f57d41ea983ea3708f7c5d1508894/src/flowstab/temporal_network.py#L232C5-L233C18

And that to save inter_Ts: https://github.com/alexbovet/flow_stability/blob/434caf46c06f57d41ea983ea3708f7c5d1508894/src/flowstab/temporal_network.py#L407

Could we use that?

For the transition matrices, I found that hickle was pretty good. but this we can do later.

[j-i-l]

Would the goal be to save inter_Ts on their own? It the idea is to simply be able to store and reload a FlosStability instance then I'd simply pickle it. Or does speak anything against this approach?

[alexbovet]

inter_Ts, and the autocov intergrals matrices can become really big, so we will need specific smart ways to save/load them. (some are already implemented in the links I gave above and also in the scripts).

But this could be done within a single "save/load? function.

[j-i-l]

Smart ways in what way?

If reducing resource usage is the goal, then a more dynamic approach of importing/exporting to disk might make sense. Maybe that the matrices reside by default on disk, but only get loaded to memory upon usage.

For majortrack, back in the days, I've implemented a lazyList that behaves like a normal list but writes the individual elements into pickles and dynamically loads them into memory upon direct access of an item, no save and load operations need to be run manually, they are all hooked to the item setters and getters:

https://github.com/t4d-gmbh/MajorTrack/blob/master/majortrack/lazylist.py#L15

Do you mean something in this direction?

[alexbovet]

I've already implemented some "smart" saving in save_inter_Ts for example.But, I need this think a bit more about how we should do that in the best way...

[j-i-l]

So would the goal be to reduce file size and storage space, like you do when storing only diffs?:

https://github.com/alexbovet/flow_stability/blob/434caf46c06f57d41ea983ea3708f7c5d1508894/src/flowstab/temporal_network.py#L491

If this is already what you want then we can simply create an interface method in FlowStability for the method you implemented:

https://github.com/alexbovet/flow_stability/blob/434caf46c06f57d41ea983ea3708f7c5d1508894/src/flowstab/temporal_network.py#L407

[j-i-l]

I've already implemented some "smart" saving in save_inter_Ts for example.But, I need this think a bit more about how we should do that in the best way...

Let me know if you have some clearer idea of how you want to do this, or what it is that we need to minimize (e.g. storage space, RAM usage, access times, ...).