guac
guac copied to clipboard
[ingestion issue] Error inserting an array as a node property in AWS Neptune using openCypher
Describe the bug
Currently Amazon Neptune does not support storing list based properties. GUAC throws the following error message while ingesting sample data with neptune as the backend - Expected a simple literal but found List.
To Reproduce Steps to reproduce the behavior:
-
Populate accurate credentials for neptune endpoint, port and region in guac.yaml
-
Build and run:
/bin/guacgql --gql-test-data=true --gql-backend=neptune
Sample logs for reference:
{"level":"error","ts":1691003568.6114063,"caller":"cmd/ingest.go:177","msg":"Error in ingesting: input: ingestSLSA internal system error\n\n","stacktrace":"github.com/guacsec/guac/cmd/guacgql/cmd.ingestSLSA\n\t/Users/smenezes/dev/guac-exp/guac/cmd/guacgql/cmd/ingest.go:177\ngithub.com/guacsec/guac/cmd/guacgql/cmd.ingestData\n\t/Users/smenezes/dev/guac-exp/guac/cmd/guacgql/cmd/ingest.go:46"}
{"level":"error","ts":1691003569.0164819,"caller":"cmd/ingest.go:265","msg":"Error in ingesting package: input: ingestPackage Neo4jError: \"BoltProtocol.unexpectedException\" (\"Unexpected server exception 'Expected a simple literal but found List.'\")\n\n","stacktrace":"github.com/guacsec/guac/cmd/guacgql/cmd.ingestDependency\n\t/Users/smenezes/dev/guac-exp/guac/cmd/guacgql/cmd/ingest.go:265\ngithub.com/guacsec/guac/cmd/guacgql/cmd.ingestData\n\t/Users/smenezes/dev/guac-exp/guac/cmd/guacgql/cmd/ingest.go:47"}
{"level":"error","ts":1691003569.51768,"caller":"cmd/ingest.go:268","msg":"Error in ingesting dependency package: input: ingestPackage Neo4jError: \"BoltProtocol.unexpectedException\" (\"Unexpected server exception 'Expected a simple literal but found List.'\")\n\n","stacktrace":"github.com/guacsec/guac/cmd/guacgql/cmd.ingestDependency\n\t/Users/smenezes/dev/guac-exp/guac/cmd/guacgql/cmd/ingest.go:268\ngithub.com/guacsec/guac/cmd/guacgql/cmd.ingestData\n\t/Users/smenezes/dev/guac-exp/guac/cmd/guacgql/cmd/ingest.go:47"}
{"level":"error","ts":1691003569.6975856,"caller":"cmd/ingest.go:271","msg":"Error in ingesting: input: ingestDependency Result contains no more records\n\n","stacktrace":"github.com/guacsec/guac/cmd/guacgql/cmd.ingestDependency\n\t/Users/smenezes/dev/guac-exp/guac/cmd/guacgql/cmd/ingest.go:271\ngithub.com/guacsec/guac/cmd/guacgql/cmd.ingestData\n\t/Users/smenezes/dev/guac-exp/guac/cmd/guacgql/cmd/ingest.go:47"}
{"level":"error","ts":1691003570.0464222,"caller":"cmd/ingest.go:265","msg":"Error in ingesting package: input: ingestPackage Neo4jError: \"BoltProtocol.unexpectedException\" (\"Unexpected server exception 'Expected a simple literal but found List.'\")\n\n","stacktrace":"github.com/guacsec/guac/cmd/guacgql/cmd.ingestDependency\n\t/Users/smenezes/dev/guac-exp/guac/cmd/guacgql/cmd/ingest.go:265\ngithub.com/guacsec/guac/cmd/guacgql/cmd.ingestData\n\t/Users/smenezes/dev/guac-exp/guac/cmd/guacgql/cmd/ingest.go:47"}
{"level":"error","ts":1691003570.571248,"caller":"cmd/ingest.go:268","msg":"Error in ingesting dependency package: input: ingestPackage Neo4jError: \"BoltProtocol.unexpectedException\" (\"Unexpected server exception 'Expected a simple literal but found List.'\")\n\n","stacktrace":"github.com/guacsec/guac/cmd/guacgql/cmd.ingestDependency\n\t/Users/smenezes/dev/guac-exp/guac/cmd/guacgql/cmd/ingest.go:268\ngithub.com/guacsec/guac/cmd/guacgql/cmd.ingestData\n\t/Users/smenezes/dev/guac-exp/guac/cmd/guacgql/cmd/ingest.go:47"}
{"level":"error","ts":1691003570.7710173,"caller":"cmd/ingest.go:271","msg":"Error in ingesting: input: ingestDependency Result contains no more records\n\n","stacktrace":"github.com/guacsec/guac/cmd/guacgql/cmd.ingestDependency\n\t/Users/smenezes/dev/guac-exp/guac/cmd/guacgql/cmd/ingest.go:271\ngithub.com/guacsec/guac/cmd/guacgql/cmd.ingestData\n\t/Users/smenezes/dev/guac-exp/guac/cmd/guacgql/cmd/ingest.go:47"}
{"level":"error","ts":1691003571.1516914,"caller":"cmd/ingest.go:265","msg":"Error in ingesting package: input: ingestPackage Neo4jError: \"BoltProtocol.unexpectedException\" (\"Unexpected server exception 'Expected a simple literal but found List.'\")\n\n","stacktrace":"github.com/guacsec/guac/cmd/guacgql/cmd.ingestDependency\n\t/Users/smenezes/dev/guac-exp/guac/cmd/guacgql/cmd/ingest.go:265\ngithub.com/guacsec/guac/cmd/guacgql/cmd.ingestData\n\t/Users/smenezes/dev/guac-exp/guac/cmd/guacgql/cmd/ingest.go:47"}
{"level":"error","ts":1691003571.6952338,"caller":"cmd/ingest.go:268","msg":"Error in ingesting dependency package: input: ingestPackage Neo4jError: \"BoltProtocol.unexpectedException\" (\"Unexpected server exception 'Expected a simple literal but found List.'\")\n\n","stacktrace":"github.com/guacsec/guac/cmd/guacgql/cmd.ingestDependency\n\t/Users/smenezes/dev/guac-exp/guac/cmd/guacgql/cmd/ingest.go:268\ngithub.com/guacsec/guac/cmd/guacgql/cmd.ingestData\n\t/Users/smenezes/dev/guac-exp/guac/cmd/guacgql/cmd/ingest.go:47"}
{"level":"error","ts":1691003571.8683348,"caller":"cmd/ingest.go:271","msg":"Error in ingesting: input: ingestDependency Result contains no more records\n\n","stacktrace":"github.com/guacsec/guac/cmd/guacgql/cmd.ingestDependency\n\t/Users/smenezes/dev/guac-exp/guac/cmd/guacgql/cmd/ingest.go:271\ngithub.com/guacsec/guac/cmd/guacgql/cmd.ingestData\n\t/Users/smenezes/dev/guac-exp/guac/cmd/guacgql/cmd/ingest.go:47"}
{"level":"error","ts":1691003572.4423685,"caller":"cmd/ingest.go:356","msg":"Error in ingesting package: input: ingestPackage Neo4jError: \"BoltProtocol.unexpectedException\" (\"Unexpected server exception 'Expected a simple literal but found List.'\")\n\n","stacktrace":"github.com/guacsec/guac/cmd/guacgql/cmd.ingestOccurrence\n\t/Users/smenezes/dev/guac-exp/guac/cmd/guacgql/cmd/ingest.go:356\ngithub.com/guacsec/guac/cmd/guacgql/cmd.ingestData\n\t/Users/smenezes/dev/guac-exp/guac/cmd/guacgql/cmd/ingest.go:48"}
{"level":"error","ts":1691003572.5802956,"caller":"cmd/ingest.go:359","msg":"Error in ingesting: input: ingestOccurrence Result contains no more records\n\n","stacktrace":"github.com/guacsec/guac/cmd/guacgql/cmd.ingestOccurrence\n\t/Users/smenezes/dev/guac-exp/guac/cmd/guacgql/cmd/ingest.go:359\ngithub.com/guacsec/guac/cmd/guacgql/cmd.ingestData\n\t/Users/smenezes/dev/guac-exp/guac/cmd/guacgql/cmd/ingest.go:48"}
{"level":"error","ts":1691003573.0326915,"caller":"cmd/ingest.go:356","msg":"Error in ingesting package: input: ingestPackage Neo4jError: \"BoltProtocol.unexpectedException\" (\"Unexpected server exception 'Expected a simple literal but found List.'\")\n\n","stacktrace":"github.com/guacsec/guac/cmd/guacgql/cmd.ingestOccurrence\n\t/Users/smenezes/dev/guac-exp/guac/cmd/guacgql/cmd/ingest.go:356\ngithub.com/guacsec/guac/cmd/guacgql/cmd.ingestData\n\t/Users/smenezes/dev/guac-exp/guac/cmd/guacgql/cmd/ingest.go:48"}
{"level":"error","ts":1691003573.1588724,"caller":"cmd/ingest.go:359","msg":"Error in ingesting: input: ingestOccurrence Result contains no more records\n\n","stacktrace":"github.com/guacsec/guac/cmd/guacgql/cmd.ingestOccurrence\n\t/Users/smenezes/dev/guac-exp/guac/cmd/guacgql/cmd/ingest.go:359\ngithub.com/guacsec/guac/cmd/guacgql/cmd.ingestData\n\t/Users/smenezes/dev/guac-exp/guac/cmd/guacgql/cmd/ingest.go:48"}
{"level":"error","ts":1691003574.333281,"caller":"cmd/ingest.go:599","msg":"Error in ingesting package: input: ingestPackage Neo4jError: \"BoltProtocol.unexpectedException\" (\"Unexpected server exception 'Expected a simple literal but found List.'\")\n\n","stacktrace":"github.com/guacsec/guac/cmd/guacgql/cmd.ingestVulnerability\n\t/Users/smenezes/dev/guac-exp/guac/cmd/guacgql/cmd/ingest.go:599\ngithub.com/guacsec/guac/cmd/guacgql/cmd.ingestData\n\t/Users/smenezes/dev/guac-exp/guac/cmd/guacgql/cmd/ingest.go:49"}
Expected behavior Ingestion should be error free and end gracefully. Since we are using neo4j driver for neptune, this might involving rewriting the neo4j queries such that list values are stored and retrieved as comma separated strings. Also rename the package to something more generic (like opencypher).
See https://docs.aws.amazon.com/neptune/latest/userguide/migration-opencypher-rewrites.html#migration-opencypher-rewrites-lists.
GUAC version Are you using a released version of GUAC or one build from a commit in the repository? main
Ingested document(s) Test data
Additional context Add any other context about the problem here.