How could the persistent store work?

[shuanzi]

Hi,All~

I'm working on a project that may need some RDF manipulation,and here i come to rdfstore-js. I'm looking for a way of persistent store for triples. I didn't find anything about it on the document. So, i read the code that mention like this:

var params = { 'persistent': true } new rdfstore.Store(params, function(err, store) { ... }

But somehow, the object store doesn't work here. In the code, it says that the persistent store is working on the indexeddb which is built on the sqlite3.

Is there anyone could give me some advice or clue for the persistent store stuff about rdfstore? That would be very helpful.

Thanks, all the best!

[csarven]

I second this question/issue. Moreover:

What I find awkward is that without some sort of a store persistence, the first query filters the result set and thereby any subsequent queries are based off that state. For one off executions, I would expect the default behaviour of the store to simply retain whatever was loaded in the store, without allowing read queries to manipulate the store.

[antoniogarrote]

Hi, a few things:

• Persistence

The new version offers persistence in the browser using IndexDB. Persistence is not provided in the server/node.js. As part of the rewrite I tried to focus in the areas where the library can be more useful, the browser. There are many options to have a persistent triple store in the server side so if that's what you are looking for probably it would be more useful for you to find a good JS client library for any of these stores.

Your code looks fine. If you have found a problem using the persistence feature in the browser, please, send me some sample code showing the problem and I will try to fix it as soon as possible.

• SQLite3

The project includes a node dependency 'indexeddb.js' that relies on sqlite3 to mimic the browser API for IndexedDB. This is just a convenience for development. The library (Indexeddb.js) is not production ready and it's actually broken, so I need either to find time to submit a patch or find a replacement. As I said providing persistence in node is currently out of the scope of the project.

• Read queries / state

Read queries do not modify the state. SELECT queries return a result set object and CONSTRUCT queries return a RDFInterfaces compatible graph object with the matching subgraph but don't modify the data in the store. The only way of changing the data is through LOAD/INSERT/DELETE/UPDATE operations. If you have found an example of that behaviour, please, send it to me and I will try to fix the bug.

Sorry for the annoyances, but I'm still working in the new version. All tests are passing and I'm trying to increase coverage as part of the rewrite, but expect some bugs to be yet to be discovered.

[JuniperChicago]

I concur with @csarven and I am hoping to bolt rdfstore-js to a key-value based store (lmdb) that stores a local copy on the file system. It is incredibly fast, immutable, and lends itself to a natural graph traversal API. I am am big fan of being able containerize a site on the server side without requiring external database connections, and rdfstore-js goes the furthest in making this happen.

I achieved my goals in the earlier version of rdfstore-js by using a MongoDB variant called Tingo that allowed me to store the database on the local file system, not requiring troublesome and slow external database connections when using Azure. But it was not ideal, and am working on a more holistic approach.

I think a persistent store based on a level-db API might provided considerably more utility and be closer to production ready. One could use a variety of browser and server based on the level-db ecosystem.

[shuanzi]

@antoniogarrote Thanks for your reponse! Actually, i'm just doing it in node.js, and maybe that's why i can't get the result. So, i better get some other library for this. And thanks again!

[muety]

I like rdfstore-js since it is the only rdfstore implementation for Javascript that supports all the things I need (JSON-LD support, SPARQL queryable, the super cool subscribe API) - the only thing I'm missing is at least a little bit of persistence when using server-sided. I would be totally pleased if there was at least kind of an export functionality to serialize the entire graph to JSON-LD (or N3 or any other format), save it to a file and import it again when starting the server for the next time. I wasn't able to do so with the current API. Is there any way to do this and if not, could anybody please implement this? I found a method called graphToJSONLD() in an older version (0.8), but expectedly it isn't anymore compatible with the current version's code at all.

[federicomorando]

Can yuo extract data with a CONSTRUCT query?

CONSTRUCT { ?s ?p ?o . }
WHERE {?s ?p ?o . }
LIMIT 100

depending on the dimension of the graph, this may be practical or not.

[muety]

I managed it by serializing the result of store.graph() (or the query @federicomorando that has mentioned - the results are the same) to JSON-LD using the server.graphToJSONLD() method from version 0.8. The problem with the plain result of store.graph() or that query was that I couldn't find a way to re-insert it to a store again, because store.load() or store.insert() both expect either JSON-LD or N3 or one of these formats, if I got it right. Anyway, I think my problem is solved now.

[UndefinedU]

Thanks for helping my wife cheat. Not that she needed encouraging but fun anyway