beam icon indicating copy to clipboard operation
beam copied to clipboard

Table with no primary key seems to cause an out of memory error during schema migration

Open mightybyte opened this issue 4 years ago • 4 comments

I'm getting the following error in what appears to be one of either verifySchema or autoMigrate.

internal error: Unable to commit 1048576 bytes of memory
    (GHC version 8.6.5 for x86_64_unknown_linux)
    Please report this as a GHC bug:  http://www.haskell.org/ghc/reportabug

We ended up finding that the problem seems to be related to having a table with no primary key. The Table instance was defined like this.

 instance Table FooT where
  data PrimaryKey FooT f = FooNoId
     deriving stock (Generic)
     deriving anyclass (Beamable)
  primaryKey _ = FooNoId

But when we changed it to have a primary key, the above memory crash went away. I don't have time to dig into the details at the moment, but I wanted to record this behavior in case anyone else runs into it.

mightybyte avatar Jul 08 '21 17:07 mightybyte

Another problem with tables that have no primary key is that the onConflictDoNothing function generates the following SQL snippet:

ON CONFLICT () DO NOTHING

The empty parentheses here are an SQL syntax error and the query fails. It looks like beam needs better support across the board for tables with no primary keys.

mightybyte avatar Jul 28 '21 14:07 mightybyte

@mightybyte regarding the upsert error, what is the semantics you intend there? Logically an empty set of columns matches any row, and should always trigger the conflict action. Do you mean to use anyConflict?

kmicklas avatar Jan 22 '22 20:01 kmicklas

Hmmm, possibly. It's been awhile. Although I would still say that beam should support tables without a primary key. Ideally (and I realize this may be a much bigger and maybe infeasible ask) things like onConflictDoNothing should have better failure modes rather than a runtime query syntax error.

mightybyte avatar Feb 03 '22 12:02 mightybyte

Agreed, upsert has a number of easy to trigger runtime failure modes that I'd like to make impossible in the next API breaking release. I also would like to at some point rewrite the solver to work by structural induction rather than graph search so that we never get these infinite / near infinite / OOM failures.

kmicklas avatar Feb 03 '22 13:02 kmicklas