pg-mem
pg-mem copied to clipboard
Issue with Sequelize "belongsToMay" association queries
Describe the bug
PG-Mem seems to have an issue parsing sequelize queries that include tables connected with "belongsToMany" association (docs here), which is the recommended way to handle many-to-many relationships (described here).
This Sequelize pattern works with our actual database, but our tests using pg-mem throw out this error:
QueryError: 💔 Your query failed to parse.
This is most likely due to a SQL syntax error. However, you might also have hit a bug, or an unimplemented feature of pg-mem.
If this is the case, please file an issue at https://github.com/oguimbal/pg-mem along with a query that reproduces this syntax error.
👉 Failed query:
SELECT "textbook"."book_id" AS "bookId", "textbook"."title", "authors"."author_id" AS "authors.authorId", "authors"."name" AS "authors.name", "authors->textbookAuthor"."book_id" AS "authors.textbookAuthor.bookId", "authors->textbookAuthor"."author_id" AS "authors.textbookAuthor.authorId" FROM "public"."textbook" AS "textbook" LEFT OUTER JOIN ( "public"."textbook_author" AS "authors->textbookAuthor" INNER JOIN "public"."author" AS "authors" ON "authors"."author_id" = "authors->textbookAuthor"."author_id") ON "textbook"."book_id" = "authors->textbookAuthor"."book_id" WHERE "textbook"."book_id" = 1;;
💀 Syntax error at line 1 col 347:
1 SELECT "textbook"."book_id" AS "bookId", "textbook"."title", "authors"."author_id" AS "authors.authorId", "authors"."name" AS "authors.name", "authors->textbookAuthor"."book_id" AS "authors.textbookAuthor.bookId", "authors->textbookAuthor"."author_id" AS "authors.textbookAuthor.authorId" FROM "public"."textbook" AS "textbook" LEFT OUTER JOIN ( "public"."textbook_author" AS "authors->textbookAuthor" INNER JOIN "public"."author" AS "authors" ON "authors"."author_id" = "authors->textbookAuthor"."author_id") ON "textbook"."book_id" = "authors->textbookAuthor"."book_id" WHERE "textbook"."book_id" = 1;;
^
Unexpected quoted_word token: "public". Instead, I was expecting to see one of the following:
- A "kw_with" token
- A "kw_select" token
- A "word" token
- A "kw_with" token
- A "lparen" token
Note: Sequelize also allows a different pattern for many-to-many, which does work despite the resulting SQL being extremely similar. This is included below as additional context.
To Reproduce
The JS code using sequelize: https://pastebin.com/paVZ0prz
The resulting SQL commands, to try in the "pg-mem playground":
-- table creation commands
CREATE TABLE IF NOT EXISTS "public"."author" ("author_id" SERIAL , "name" VARCHAR(255) NOT NULL, PRIMARY KEY ("author_id"));
CREATE TABLE IF NOT EXISTS "public"."textbook" ("book_id" SERIAL , "title" VARCHAR(255) NOT NULL, PRIMARY KEY ("book_id"));
CREATE TABLE IF NOT EXISTS "public"."textbook_author" ("book_id" INTEGER NOT NULL REFERENCES "public"."textbook" ("book_id") ON DELETE CASCADE ON UPDATE CASCADE, "author_id" INTEGER NOT NULL REFERENCES "public"."author" ("author_id") ON DELETE CASCADE ON UPDATE CASCADE, UNIQUE ("book_id", "author_id"), PRIMARY KEY ("book_id","author_id"));
-- Basic data inserts
INSERT INTO "public"."author" ("author_id","name") VALUES (DEFAULT,'John Doe'),(DEFAULT,'Einstein'),(DEFAULT,'Steve'),(DEFAULT,'Bill') RETURNING "author_id","name";
INSERT INTO "public"."textbook" ("book_id","title") VALUES (DEFAULT,'Science 101'),(DEFAULT,'Math 101'),(DEFAULT,'World History 101') RETURNING "book_id","title";
INSERT INTO "public"."textbook_author" ("book_id","author_id") VALUES (1,1),(1,2),(1,3) RETURNING "book_id","author_id";
-- The failing command:
SELECT "textbook"."book_id" AS "bookId", "textbook"."title", "authors"."author_id" AS "authors.authorId", "authors"."name" AS "authors.name", "authors->textbookAuthor"."book_id" AS "authors.textbookAuthor.bookId", "authors->textbookAuthor"."author_id" AS "authors.textbookAuthor.authorId" FROM "public"."textbook" AS "textbook" LEFT OUTER JOIN ( "public"."textbook_author" AS "authors->textbookAuthor" INNER JOIN "public"."author" AS "authors" ON "authors"."author_id" = "authors->textbookAuthor"."author_id") ON "textbook"."book_id" = "authors->textbookAuthor"."book_id" WHERE "textbook"."book_id" = 1;
pg-mem version
2.9.1, but I can replicate this with 3.0.5 and the sandbox
Additional Context
- Sequelize also allows many-to-many by defining 2 one-to-many relationships instead, as described here. This does work, but it is less ideal to use and the output is more cumbersome.
- Using this pattern, the resulting SQL correctly parsed query would be:
SELECT "textbook"."book_id" AS "bookId", "textbook"."title", "textbookAuthors"."book_id" AS "textbookAuthors.bookId", "textbookAuthors"."author_id" AS "textbookAuthors.authorId", "textbookAuthors->author"."author_id" AS "textbookAuthors.author.authorId", "textbookAuthors->author"."name" AS "textbookAuthors.author.name" FROM "public"."textbook" AS "textbook" LEFT OUTER JOIN "public"."textbook_author" AS "textbookAuthors" ON "textbook"."book_id" = "textbookAuthors"."book_id" LEFT OUTER JOIN "public"."author" AS "textbookAuthors->author" ON "textbookAuthors"."author_id" = "textbookAuthors->author"."author_id" WHERE "textbook"."book_id" = 1;
- The JS code for this query was:
author.hasMany(textbookAuthor, { foreignKey: 'authorId' });
textbookAuthor.belongsTo(author, { foreignKey: 'authorId' });
textbook.hasMany(textbookAuthor, { foreignKey: 'bookId' });
textbookAuthor.belongsTo(textbook, { foreignKey: 'bookId' });
await textbook.findOne({
where: { bookId: 1 },
include: {
model: sequelize.models.textbookAuthor,
include: { model: sequelize.models.author },
},
});
- Based on the errors and the working/not-working SQL commands, I'm guessing it may be a parsing issue with how pg-mem handles either the "join" or the "->" json objects. That is just a guess, though.
- The "2 one-to-many" pattern also fails if the nested include has "required: true" while the outer include does not, but any other combination does not throw an error. This results in the SQL having "LEFT OUTER JOIN(... INNER JOIN...)", but any other combination (like RIGHT joins or both INNER joins) appear to work fine.
SELECT "textbook"."book_id" AS "bookId", "textbook"."title", "textbookAuthors"."book_id" AS "textbookAuthors.bookId", "textbookAuthors"."author_id" AS "textbookAuthors.authorId", "textbookAuthors->author"."author_id" AS "textbookAuthors.author.authorId", "textbookAuthors->author"."name" AS "textbookAuthors.author.name" FROM "public"."textbook" AS "textbook" LEFT OUTER JOIN ( "public"."textbook_author" AS "textbookAuthors" INNER JOIN "public"."author" AS "textbookAuthors->author" ON "textbookAuthors"."author_id" = "textbookAuthors->author"."author_id" ) ON "textbook"."book_id" = "textbookAuthors"."book_id" WHERE "textbook"."book_id" = 1;
- The JS code that causes this error:
await textbook.findOne({
where: { bookId: 1 },
include: {
model: textbookAuthor,
include: {
model: author,
required: true, // this only breaks if the "textbookAuthor" isn't also "required: true"
},
},
});
- After some searching, this but may be related to these other issues (but not certain):
- https://github.com/oguimbal/pg-mem/issues/463
- https://github.com/oguimbal/pg-mem/issues/299
- https://github.com/oguimbal/pg-mem/issues/444