datahub-gma icon indicating copy to clipboard operation
datahub-gma copied to clipboard

Add support for Gremlin

Open mhaley-miovision opened this issue 3 years ago • 2 comments

Is your feature request related to a problem? Please describe.

We are considering using an upstream project datahub. Our team is an AWS shop, and would like to take advantage of AWS hosted solutions like Neptune whenever possible. It would be great to add support for Gremlin (one of the interfaces that Neptune implements) so that we can easily host the graph database for datahub.

Describe the solution you'd like

Implement a BaseGraphWriterDAO and BaseQueryDAO for gremlin based graph data stores.

Describe alternatives you've considered

Alternatives would be finding 3rd party neo4j SaaS provider, or hosting our own database cluster within AWS. Both of these are something that we would prefer to avoid if possible, for both cost and business reasons.

Additional context

Add any other context or screenshots about the feature request here.

mhaley-miovision avatar Apr 08 '21 20:04 mhaley-miovision

Sounds like a great idea. @mhaley-miovision : are you thinking of contributing this as well?

shirshanka avatar Apr 08 '21 20:04 shirshanka

Thanks for the interest @mhaley-miovision. Did you have a chance to spend some time on the implementation of this idea?

We have explored this last year and implemented a Gremlin DAO internally at Linkedin. However, there were some blockers along the way and couldn't proceed further.

  1. We couldn't use neo4j-gremlin maintained by Neo4j the company because of the AGPL license. Refer to here: http://tinkerpop.apache.org/docs/3.3.4-SNAPSHOT/reference/#neo4j-gremlin
Unless under a commercial agreement with Neo4j, Inc., Neo4j is licensed AGPL. The neo4j-gremlin module is licensed Apache2 because it only references the Apache2-licensed Neo4j API (not its implementation).

<!-- neo4j-tinkerpop-api-impl is NOT Apache 2 licensed - more information below -->
  1. We tried to use another 3rd party library (https://github.com/SteelBridgeLabs/neo4j-gremlin-bolt) but that also seemed half cooked and didn't work out well for us. Even the link doesn't work right now :)

So, in summary, although we have an implementation, we couldn't quite use that in production. Therefore, we didn't share it with the community.

I would like to collaborate and hear more from you if you have other ideas for this.

keremsahin1 avatar May 25 '21 19:05 keremsahin1