tortoise-orm icon indicating copy to clipboard operation
tortoise-orm copied to clipboard

bulk_update fails when a jsonb field contains an array with a single quote entry under postgres

Open AstraliteHeart opened this issue 3 years ago • 3 comments

Describe the bug Updating a jsonb field via bulk_update with an array containing string values with single quotes, the resulting SQL is incorrect due to lack of escaping which leads to the update failure.

To Reproduce

I have not yet verified if this is relevant, but in my application code two conditions are present:

  1. I am updating an entry previously fetched form a database
  2. Field is updated to a top level array
class MemeModel(Model):
    tags = fields.JSONField()

meme = MemeModel.all().first()
meme.tags = ["it's friday my dudes"]
MemeModel.bulk_update([meme], fields=['tags'])
Traceback (most recent call last):
  File "tortoise\backends\asyncpg\client.py", line 36, in translate_exceptions_
    return await func(self, *args)
  File "tortoise\backends\asyncpg\client.py", line 178, in execute_query
    res = await connection.execute(*params)
  File "asyncpg\connection.py", line 318, in execute
    return await self._protocol.query(query, timeout)
  File "asyncpg\protocol\protocol.pyx", line 338, in query
asyncpg.exceptions.PostgresSyntaxError: syntax error at or near "s"
... "tags"=CASE WHEN "id"=123 THEN '["it's friday my dudes"]'::jsonb END...

My understanding is that while json.dumps escapes double quotes, the single quote is still present un-escaped in the final string and conflicts with the ::jsonb cast which also uses single quotes. I suspect the issue may be mitigated either by using $$ for quoting or with an extra pass to escape the single quote within the result of json.dumps

Expected behavior bulk_update succeeds

Additional context I've verified that https://github.com/tortoise/tortoise-orm/pull/1023 (and the follow up commit https://github.com/tortoise/tortoise-orm/commit/1c8d30466b2a82914ccc731f6ac0d5755b682f4f + pypika git branch do not fix the issue).

AstraliteHeart avatar Feb 16 '22 03:02 AstraliteHeart

Try meme.tags = ["it''s friday my dudes"]

long2ice avatar Feb 16 '22 03:02 long2ice

Works great! But I assume this also means data loss for cases where '' was in the original data for whatever reason, right?

AstraliteHeart avatar Feb 16 '22 04:02 AstraliteHeart

I got the same bug.

tortoise.exceptions.OperationalError: column "xxx" is of type jsonb but expression is of type text[]

Achilles0509 avatar Apr 08 '22 14:04 Achilles0509