Query with a lot of conditions causes StackOverflowError

[takezoe]

With Slick 3.1.1, I got StackOverflowError for a query which has a lot of conditions.

Code to reproduce easily is below:

val seq = Range(1, 500)
Accounts.filter { t => seq.map { x => t.userId === x.bind }.reduce(_ || _)}.list

An error caused by above query is:

java.lang.StackOverflowError
    at scala.collection.immutable.Set$class.seq(Set.scala:55)
    at scala.collection.immutable.Set$EmptySet$.seq(Set.scala:68)
    at scala.collection.immutable.Set$EmptySet$.seq(Set.scala:68)
    at scala.collection.SetLike$class.$plus$plus(SetLike.scala:141)
    at scala.collection.AbstractSet.$plus$plus(Set.scala:47)
    at slick.compiler.ExpandSums.slick$compiler$ExpandSums$$tr$1(ExpandSums.scala:27)
    at slick.compiler.ExpandSums$$anonfun$7.apply(ExpandSums.scala:32)
    at slick.compiler.ExpandSums$$anonfun$7.apply(ExpandSums.scala:32)
    at slick.util.ConstArray.endoMap(ConstArray.scala:122)
    at slick.ast.Node$class.mapChildren(Node.scala:51)
    at slick.ast.Apply.mapChildren(Node.scala:539)
    at slick.compiler.ExpandSums.slick$compiler$ExpandSums$$tr$1(ExpandSums.scala:32)
    at slick.compiler.ExpandSums$$anonfun$7.apply(ExpandSums.scala:32)
    at slick.compiler.ExpandSums$$anonfun$7.apply(ExpandSums.scala:32)
    at slick.util.ConstArray.endoMap(ConstArray.scala:122)
    at slick.ast.Node$class.mapChildren(Node.scala:51)
    at slick.ast.Apply.mapChildren(Node.scala:539)

I tried decreasing condition. Then I got another StackOverflowError.

val seq = Range(1, 200)
Accounts.filter { t => seq.map { x => t.userId === x.bind }.reduce(_ || _)}.list

An error caused by above query is:

java.lang.StackOverflowError
    at slick.driver.JdbcStatementBuilderComponent$QueryBuilder.expr(JdbcStatementBuilderComponent.scala:306)
    at slick.driver.PostgresDriver$QueryBuilder.expr(PostgresDriver.scala:164)
    at slick.driver.JdbcStatementBuilderComponent$QueryBuilder$$anonfun$expr$7.apply(JdbcStatementBuilderComponent.scala:376)
    at slick.driver.JdbcStatementBuilderComponent$QueryBuilder$$anonfun$expr$7.apply(JdbcStatementBuilderComponent.scala:376)
    at slick.util.SQLBuilder.sep(SQLBuilder.scala:31)
    at slick.driver.JdbcStatementBuilderComponent$QueryBuilder.expr(JdbcStatementBuilderComponent.scala:376)
    at slick.driver.PostgresDriver$QueryBuilder.expr(PostgresDriver.scala:164)
    at slick.driver.JdbcStatementBuilderComponent$QueryBuilder$$anonfun$expr$7.apply(JdbcStatementBuilderComponent.scala:376)
    at slick.driver.JdbcStatementBuilderComponent$QueryBuilder$$anonfun$expr$7.apply(JdbcStatementBuilderComponent.scala:376)
    at slick.util.SQLBuilder.sep(SQLBuilder.scala:31)

[raam86]

+1

[mycentrio]

+1

[spilliton]

Samesies

[takezoe]

I found a workaround that replacing consecutive ORs with IN clause as below:

val seq = Range(1, 500)

// This causes StackOverflowError
Accounts.filter { t => seq.map { x => t.userId === x.bind }.reduce(_ || _)}.list

// This works
Accounts.filter { t => t.userId inSetBind(seq) }.list

However this isn't a perfect workaround because not all queries can be rewritten with IN clause. Also some databases have the limitation of number of IN parameters.

[dominics]

However this isn't a perfect workaround because not all queries can be rewritten with IN clause

Specifically, #517 means compound primary keys can't be queried with inSet

[grillorafael]

I am having the same issue but I can't use inSet since I have to do this based on 2 attributes.

Had to use raw sql (which is not great) to sort it out

[teeterc]

For a workaround: Group the seq.

 query.filter(t => seq.grouped(10).map(_.map{
          s =>
               t.a === s.a && t.b === s.b
        }).foldLeft(false.bind)(_ || _)
   .foldLeft(false.bind)(_ || _)
)

[fgfernandez0321]

@teeterc We can create balanced groups using Math.sqrt. It works for 10000 items or less.

    val groupSize = Math.max(Math.sqrt(seq.size).toInt, 1)

    query.filter { row =>
      seq
        .grouped(groupSize)
        .map(
          _.map { case (a, b) => row.a === a && row.b === b }
            .reduceLeftOption(_ || _)
            .getOrElse(false: Rep[Boolean])
        )
        .foldLeft(false.bind)(_ || _)
    }.result