guac icon indicating copy to clipboard operation
guac copied to clipboard

[ingestion issue] Error inserting an array as a node property in AWS Neptune using openCypher

Open stevemenezes opened this issue 2 years ago • 0 comments

Describe the bug Currently Amazon Neptune does not support storing list based properties. GUAC throws the following error message while ingesting sample data with neptune as the backend - Expected a simple literal but found List.

To Reproduce Steps to reproduce the behavior:

  1. Populate accurate credentials for neptune endpoint, port and region in guac.yaml

  2. Build and run: /bin/guacgql --gql-test-data=true --gql-backend=neptune

Sample logs for reference:

{"level":"error","ts":1691003568.6114063,"caller":"cmd/ingest.go:177","msg":"Error in ingesting: input: ingestSLSA internal system error\n\n","stacktrace":"github.com/guacsec/guac/cmd/guacgql/cmd.ingestSLSA\n\t/Users/smenezes/dev/guac-exp/guac/cmd/guacgql/cmd/ingest.go:177\ngithub.com/guacsec/guac/cmd/guacgql/cmd.ingestData\n\t/Users/smenezes/dev/guac-exp/guac/cmd/guacgql/cmd/ingest.go:46"}
{"level":"error","ts":1691003569.0164819,"caller":"cmd/ingest.go:265","msg":"Error in ingesting package: input: ingestPackage Neo4jError: \"BoltProtocol.unexpectedException\" (\"Unexpected server exception 'Expected a simple literal but found List.'\")\n\n","stacktrace":"github.com/guacsec/guac/cmd/guacgql/cmd.ingestDependency\n\t/Users/smenezes/dev/guac-exp/guac/cmd/guacgql/cmd/ingest.go:265\ngithub.com/guacsec/guac/cmd/guacgql/cmd.ingestData\n\t/Users/smenezes/dev/guac-exp/guac/cmd/guacgql/cmd/ingest.go:47"}
{"level":"error","ts":1691003569.51768,"caller":"cmd/ingest.go:268","msg":"Error in ingesting dependency package: input: ingestPackage Neo4jError: \"BoltProtocol.unexpectedException\" (\"Unexpected server exception 'Expected a simple literal but found List.'\")\n\n","stacktrace":"github.com/guacsec/guac/cmd/guacgql/cmd.ingestDependency\n\t/Users/smenezes/dev/guac-exp/guac/cmd/guacgql/cmd/ingest.go:268\ngithub.com/guacsec/guac/cmd/guacgql/cmd.ingestData\n\t/Users/smenezes/dev/guac-exp/guac/cmd/guacgql/cmd/ingest.go:47"}
{"level":"error","ts":1691003569.6975856,"caller":"cmd/ingest.go:271","msg":"Error in ingesting: input: ingestDependency Result contains no more records\n\n","stacktrace":"github.com/guacsec/guac/cmd/guacgql/cmd.ingestDependency\n\t/Users/smenezes/dev/guac-exp/guac/cmd/guacgql/cmd/ingest.go:271\ngithub.com/guacsec/guac/cmd/guacgql/cmd.ingestData\n\t/Users/smenezes/dev/guac-exp/guac/cmd/guacgql/cmd/ingest.go:47"}
{"level":"error","ts":1691003570.0464222,"caller":"cmd/ingest.go:265","msg":"Error in ingesting package: input: ingestPackage Neo4jError: \"BoltProtocol.unexpectedException\" (\"Unexpected server exception 'Expected a simple literal but found List.'\")\n\n","stacktrace":"github.com/guacsec/guac/cmd/guacgql/cmd.ingestDependency\n\t/Users/smenezes/dev/guac-exp/guac/cmd/guacgql/cmd/ingest.go:265\ngithub.com/guacsec/guac/cmd/guacgql/cmd.ingestData\n\t/Users/smenezes/dev/guac-exp/guac/cmd/guacgql/cmd/ingest.go:47"}
{"level":"error","ts":1691003570.571248,"caller":"cmd/ingest.go:268","msg":"Error in ingesting dependency package: input: ingestPackage Neo4jError: \"BoltProtocol.unexpectedException\" (\"Unexpected server exception 'Expected a simple literal but found List.'\")\n\n","stacktrace":"github.com/guacsec/guac/cmd/guacgql/cmd.ingestDependency\n\t/Users/smenezes/dev/guac-exp/guac/cmd/guacgql/cmd/ingest.go:268\ngithub.com/guacsec/guac/cmd/guacgql/cmd.ingestData\n\t/Users/smenezes/dev/guac-exp/guac/cmd/guacgql/cmd/ingest.go:47"}
{"level":"error","ts":1691003570.7710173,"caller":"cmd/ingest.go:271","msg":"Error in ingesting: input: ingestDependency Result contains no more records\n\n","stacktrace":"github.com/guacsec/guac/cmd/guacgql/cmd.ingestDependency\n\t/Users/smenezes/dev/guac-exp/guac/cmd/guacgql/cmd/ingest.go:271\ngithub.com/guacsec/guac/cmd/guacgql/cmd.ingestData\n\t/Users/smenezes/dev/guac-exp/guac/cmd/guacgql/cmd/ingest.go:47"}
{"level":"error","ts":1691003571.1516914,"caller":"cmd/ingest.go:265","msg":"Error in ingesting package: input: ingestPackage Neo4jError: \"BoltProtocol.unexpectedException\" (\"Unexpected server exception 'Expected a simple literal but found List.'\")\n\n","stacktrace":"github.com/guacsec/guac/cmd/guacgql/cmd.ingestDependency\n\t/Users/smenezes/dev/guac-exp/guac/cmd/guacgql/cmd/ingest.go:265\ngithub.com/guacsec/guac/cmd/guacgql/cmd.ingestData\n\t/Users/smenezes/dev/guac-exp/guac/cmd/guacgql/cmd/ingest.go:47"}
{"level":"error","ts":1691003571.6952338,"caller":"cmd/ingest.go:268","msg":"Error in ingesting dependency package: input: ingestPackage Neo4jError: \"BoltProtocol.unexpectedException\" (\"Unexpected server exception 'Expected a simple literal but found List.'\")\n\n","stacktrace":"github.com/guacsec/guac/cmd/guacgql/cmd.ingestDependency\n\t/Users/smenezes/dev/guac-exp/guac/cmd/guacgql/cmd/ingest.go:268\ngithub.com/guacsec/guac/cmd/guacgql/cmd.ingestData\n\t/Users/smenezes/dev/guac-exp/guac/cmd/guacgql/cmd/ingest.go:47"}
{"level":"error","ts":1691003571.8683348,"caller":"cmd/ingest.go:271","msg":"Error in ingesting: input: ingestDependency Result contains no more records\n\n","stacktrace":"github.com/guacsec/guac/cmd/guacgql/cmd.ingestDependency\n\t/Users/smenezes/dev/guac-exp/guac/cmd/guacgql/cmd/ingest.go:271\ngithub.com/guacsec/guac/cmd/guacgql/cmd.ingestData\n\t/Users/smenezes/dev/guac-exp/guac/cmd/guacgql/cmd/ingest.go:47"}
{"level":"error","ts":1691003572.4423685,"caller":"cmd/ingest.go:356","msg":"Error in ingesting package: input: ingestPackage Neo4jError: \"BoltProtocol.unexpectedException\" (\"Unexpected server exception 'Expected a simple literal but found List.'\")\n\n","stacktrace":"github.com/guacsec/guac/cmd/guacgql/cmd.ingestOccurrence\n\t/Users/smenezes/dev/guac-exp/guac/cmd/guacgql/cmd/ingest.go:356\ngithub.com/guacsec/guac/cmd/guacgql/cmd.ingestData\n\t/Users/smenezes/dev/guac-exp/guac/cmd/guacgql/cmd/ingest.go:48"}
{"level":"error","ts":1691003572.5802956,"caller":"cmd/ingest.go:359","msg":"Error in ingesting: input: ingestOccurrence Result contains no more records\n\n","stacktrace":"github.com/guacsec/guac/cmd/guacgql/cmd.ingestOccurrence\n\t/Users/smenezes/dev/guac-exp/guac/cmd/guacgql/cmd/ingest.go:359\ngithub.com/guacsec/guac/cmd/guacgql/cmd.ingestData\n\t/Users/smenezes/dev/guac-exp/guac/cmd/guacgql/cmd/ingest.go:48"}
{"level":"error","ts":1691003573.0326915,"caller":"cmd/ingest.go:356","msg":"Error in ingesting package: input: ingestPackage Neo4jError: \"BoltProtocol.unexpectedException\" (\"Unexpected server exception 'Expected a simple literal but found List.'\")\n\n","stacktrace":"github.com/guacsec/guac/cmd/guacgql/cmd.ingestOccurrence\n\t/Users/smenezes/dev/guac-exp/guac/cmd/guacgql/cmd/ingest.go:356\ngithub.com/guacsec/guac/cmd/guacgql/cmd.ingestData\n\t/Users/smenezes/dev/guac-exp/guac/cmd/guacgql/cmd/ingest.go:48"}
{"level":"error","ts":1691003573.1588724,"caller":"cmd/ingest.go:359","msg":"Error in ingesting: input: ingestOccurrence Result contains no more records\n\n","stacktrace":"github.com/guacsec/guac/cmd/guacgql/cmd.ingestOccurrence\n\t/Users/smenezes/dev/guac-exp/guac/cmd/guacgql/cmd/ingest.go:359\ngithub.com/guacsec/guac/cmd/guacgql/cmd.ingestData\n\t/Users/smenezes/dev/guac-exp/guac/cmd/guacgql/cmd/ingest.go:48"}
{"level":"error","ts":1691003574.333281,"caller":"cmd/ingest.go:599","msg":"Error in ingesting package: input: ingestPackage Neo4jError: \"BoltProtocol.unexpectedException\" (\"Unexpected server exception 'Expected a simple literal but found List.'\")\n\n","stacktrace":"github.com/guacsec/guac/cmd/guacgql/cmd.ingestVulnerability\n\t/Users/smenezes/dev/guac-exp/guac/cmd/guacgql/cmd/ingest.go:599\ngithub.com/guacsec/guac/cmd/guacgql/cmd.ingestData\n\t/Users/smenezes/dev/guac-exp/guac/cmd/guacgql/cmd/ingest.go:49"}

Expected behavior Ingestion should be error free and end gracefully. Since we are using neo4j driver for neptune, this might involving rewriting the neo4j queries such that list values are stored and retrieved as comma separated strings. Also rename the package to something more generic (like opencypher).

See https://docs.aws.amazon.com/neptune/latest/userguide/migration-opencypher-rewrites.html#migration-opencypher-rewrites-lists.

GUAC version Are you using a released version of GUAC or one build from a commit in the repository? main

Ingested document(s) Test data

Additional context Add any other context about the problem here.

stevemenezes avatar Aug 04 '23 18:08 stevemenezes