Save the rdflib graph as hdt format

[meisyarahd]

Is it possible to load my RDF file with rdflib and convert it into an hdt document, and eventually save it as a file?

[FlorianLudwig]

Hi @meisyarahd,

this is not supported with RDFLib. The model how rdflib works is incompatible with the unmutable nature of hdt-files. Therefor this is not supported. To convert to hdt you are better served using one of those tools: http://www.rdfhdt.org/downloads/

[white-gecko]

Wouldn't it be possible to implement it as a serialization, just similar to turtle?

[FlorianLudwig]

@white-gecko yes, that would be possible.

To be honest, for me this use-case is still not really interesting. Generating hdt is more resource intensive than other formats as it is heavily optimized towards reading - not writing. To generate a hdt the full graph must be sorted. As this is quite memory intensive I would always prefer to use the pure c++ implementation and avoid any overhead to generate the hdt.

And for small graphs - where the overhead does not matter - hdt might not be interesting at all.

This is not meant as a "this is a bad idea" but more as a warning to keep memory usage in mind if someone wants to take this on :)

[mielvds]

Can we not merge https://github.com/Callidon/pyHDT/pull/4? I think it's up to the user of RDFlib to think about memory, with probably the exception of threading / multiprocessing.

[chiarcos]

As a side-note, I found it relatively easy to integrate HDT-CPP via docker into Python. Just create a turtle tempfile and hand the result over to HDT-CPP.