ibis icon indicating copy to clipboard operation
ibis copied to clipboard

feat(api): generate Python code from an ibis expression

Open jcrist opened this issue 3 years ago • 1 comments

Given an ibis expression, it'd be useful to be able to generate Python code that results in the same expression.

Example user session:

import ibis


con = ibis.sqlite.connect("geography.db")

t = con.tables["countries"]

asian_countries = t['name', 'continent', 'population'].filter(t.continent == 'AS')
result = asian_countries.sort_by((asian_countries.population, False)).limit(10)


code = ibis.util.to_python_code(result)
print(code)
# output
"""
countries = con.tables["countries"]

temp1 = countries['name', 'continent', 'population'].filter(countries["continent"] == 'AS')
result = temp1.sort_by((temp1.population, False)).limit(10)
"""

Notes:

  • Given an expression, we can know the table names involved, but not the connection variable name. A good default behavior here might be to default to con, and make the connection name configurable via connection_varname="foo".
  • We can generate readable variable names for initial tables based on the table name
  • We will sometimes need to generate intermediate expressions. Here I go with temp%d with an incrementing suffix
  • We need to provide a variable name for the output expression. Here I use result - this could also be configurable, idk

I don't expect this to be used in a 100% automated fashion, rather I expect this to be a useful tool with a human-in-the-loop. I also don't expect the generated code to match 1:1 with how a human would write it. I view this mostly as a starting point that a human can then edit further.

Possible use cases:

  • Given #4324, we could then print out ibis code for generating those expressions.
  • Sometimes in an interactive session you work non-linearly, defining and redefining temporary variables until you get the expected result. It may be tricky to backtrack back to how you came up with the correct expression, but since you have the full expression tree ibis should be able to output some example code that would regenerate it.

jcrist avatar Aug 09 '22 16:08 jcrist

I think this is a great idea! I would like it if the output were at least nominally executable immediately; like if called with exec() it defined a well-known function that eval() could call to return the expression.

saulpw avatar Aug 09 '22 17:08 saulpw