ibis icon indicating copy to clipboard operation
ibis copied to clipboard

bug: `RelationError` when relating two scalar values from different relations

Open NickCrews opened this issue 1 year ago • 4 comments

What happened?

I feel like I already filed a bug for this, but I couldn't find it with some searching? Apologies if so. Thanks for the help!

import ibis
ibis.options.interactive = True

t = ibis.examples.penguins.fetch()
t.distinct().count() == t.count()

results in RelationError: Selection expressions don't fully originate from dependencies of the table expression.

t.distinct().count() and t.count() are both scalar values, so they should be able to be compared, even though they are coming from different relations. If we were comparing columns, eg t.distinct().my_col == t.my_col, then it makes sense to get this error because how would you "line up" these two columns.

What version of ibis are you using?

7.1.0

What backend(s) are you using, if any?

No response

Relevant log output

No response

Code of Conduct

  • [X] I agree to follow this project's Code of Conduct

NickCrews avatar Nov 25 '23 19:11 NickCrews

Do you have a general use case for this? It would be helpful to understand the context in which you're trying to do this.

For the query you're showing here you should be able to write

t.nunique() == t.count()

cpcloud avatar Dec 07 '23 19:12 cpcloud

A few ideas that I have had to do:

  • t1.count() == t2.count() (where unlike in the original example t1 and t2 are totally unrelated eg from different parquet files)
  • ibis.greatest(t1.ids.max(), t2.ids.max())
  • t2.mutate(ids=t1.ids.max() + ibis.row_number() + 1) (this is relating scalar from t1 to a vector in t2, so a little different, but related)

NickCrews avatar Dec 08 '23 00:12 NickCrews

Just to report, this on main now brings up a different error. More specifically a RecursionError.

    293 has_unbound = False
    294 node_types = (ops.UnboundTable, ops.DatabaseTable, ops.SQLQueryResult)
--> 295 for table in self.op().find(node_types):
    296     if isinstance(table, ops.UnboundTable):
    297         has_unbound = True

RecursionError: maximum recursion depth exceeded

that being said, I'm not sure if this is the expected bug or there is something else going on here.

ncclementi avatar Apr 04 '24 20:04 ncclementi

The infinite recursion is definitely unintended 😅

cpcloud avatar Apr 04 '24 20:04 cpcloud