agensgraph icon indicating copy to clipboard operation
agensgraph copied to clipboard

Export partial data of graph

Open imsagraph opened this issue 5 years ago • 4 comments

Hi,

I use agensgraph in version 1.3. I want to extract production data but a sample extract data. And recreate graph in a new developpement database. Can you give me some way please.

Regards.

imsagraph avatar Aug 14 '18 06:08 imsagraph

You can export nodes and relationships in 2 tables. Export tables in a classical postgresql way, and recreate graph database from the two tables.

ontologiae avatar Aug 17 '18 10:08 ontologiae

@ontologiae : Do you mean ag_edge and ag_vertex? it's not the right path because it's legacy tables, so they do not have data.

@bitnine : After many tests, I encounter a problem with the pg_dump (agens) because when exporting a schema it loads the data of all the schema of the database to put in the ag_label table of pg_catalog of database. While I need only data ag_label of the exported schema .

imsagraph avatar Aug 17 '18 14:08 imsagraph

Exactly. A table for edge and one other for vertex, and in both tables you have a jsonb field where you put your data for each vertex or edge.

If you use pg_dump, it make sense that postgresql dump all your data, it's the pg_dump function to do that.

NB : there's a lot of question here of peoples who forget that it's just a modified PostgreSQL database with cypher implementation, that's all. So just use it as a regular PostgreSQL database.

ontologiae avatar Sep 02 '18 12:09 ontologiae

Our case is very similar to the issue of the op. But the above mentioned solutions are not applicable, since the target database it not empty.

We have 2 independent AgensGraph instances, A and B. A runs with Agensgraph 1.4, B runs with AgensGraph 2.1.1.

We want to export partial data from A and import into B.

The amount of the data to export and import is huge, almost 100 million objects of vertices and edges in total.

B is not empty. B contains the exact same labels as A, but label OIDs are diffent.

The auto-generated IDs of A and B can overlap if we copy the IDs as well as the properties, however IDs of the vertices should be copied to be able to import the edges.

Using COPY would corrupt the target database badly because of the label OIDs and the unchanged table sequence values.

Is there any tool we can use for the data migration in this case?

peter-ungvari avatar Jul 01 '19 15:07 peter-ungvari