Memory Backend Implementation

[balint42]

We should discuss what the rationale behind a memory backend would be to decide about its implementation.

The "memory" characteristic could serve two features: -speed -easier to implement if done from scratch

it would IMHO not serve: -usability as "simulated dialog" / as storage for synchronization attempts

it would negatively affect: -thread safety

Remarks: I do not think that temp-storing data somewhere (like a simulated dialog form or a temp copy for synchronization) requires having it in the memory for other reasons than speed. In our special context in fact we could easily use any backend implementation to temp-store another (e.g. "HDF5" backend stores "POSTGRES" backend). Regarding thread safety: having a HDF5 file on disk is safer than having just a FILE frontend object in memory, which is thread specific currently.

All this just as prelude for the question: how should our "memory" backend look like? Should it really be a "memory" storage? If so, jgrewe proposed a very interesting option: does HDF5 support storing data in memory? In this case one could simply re-use the HDF5 backend. Disadvantage: not having an independent test backend that e.g. circumvents the HDF5 specific getter, setter, "group", "hydra" and hdf5-structures.

What are your thoughts?

[balint42]

Discussing this with Adrian @ gnode headquarters ;), we agreed that a separate, built from scratch memory backend would make sense to have a combined "speed"-"functional check" tool for other backends. It would not be thread safe, but easy to implement as follows:

-store all data in the backend entities themselves, no file layer as in HDF5 backend -restrict number of backend entities instantiated to one per entity id and thread: have an id-sharedpointer-hashmap in each backend entity that retrieves already created backend entities for a given id and binds them to a new frontend entity. Restrict relationship between frontend:backend:file by this to n : 1 : 1 per id, where in the HDF5 backend it is n : m : 1. -Weak & shared pointer: in entities which store "parent" references circular referencing might arise since all entities will store "child" references (i.e. child & parent referencing each other). Solution might be to use weak pointers. -Try to make backend ctors not easily callable (since backend entities shall be created through factories). In HDF5 backend this is given by the fact that most ctors require types that would be complicated to create manually, e.g. "group".

Notes: "file" in the above relationship in the memory backend does not really exist as it is stored & merged into the backend entity itself. The frontend objects will still be created each time "get" is called and thus "zombie" objects ( #138 ) will still exist - except if we actually destroy the backend entity once a delete is called.