triplestore icon indicating copy to clipboard operation
triplestore copied to clipboard

Would HDT be considered for a binary format?

Open fils opened this issue 7 years ago • 6 comments

http://www.rdfhdt.org/

I honestly don't know if HDT would be a benefit or a hindrance. I'm really just passing this along as an FYI to the developers.

fils avatar Sep 16 '17 13:09 fils

Looks great indeed. Thanks. I will definitely have a look. Given the format's properties we should implement it or a similar one.

simcap avatar Sep 16 '17 15:09 simcap

I think Joseki took a stab at this: https://github.com/Callidon/joseki/blob/master/graph/treeGraph.go

fils avatar Sep 18 '17 08:09 fils

Side note: Unfortunately I cannot give a deadline on when it will be done. I might do the JSON-LD implementation before and the support of blank nodes / languague tag on the custom binary format (for full compatibility)

simcap avatar Sep 21 '17 10:09 simcap

Not sure if the right place to ask, but was wondering if something similar is already possible with wallix/triplestore: Query big datasets on disk, based on a disk based index? (Having a 12GB .hdt file with a 8GB index that I'd like to work with in Golang if possible :) ).

In any case, HDT seems like a great path, as a standardized format for this.

samuell avatar Nov 17 '17 14:11 samuell

@samuell HDT is a great path indeed and this should be an option for this triplestore. Unfortunately, I cannot devote time at the moment to work on it.

To ease the task, we might cut the work in different pieces. For instance, we could start by only implementing a reader/decoder for this format.

I hope to do so in the next coming 3 months.

simcap avatar Nov 20 '17 09:11 simcap

@simcap IC, thanks for the status update!

samuell avatar Nov 20 '17 14:11 samuell