feat(decompile): support rendering ibis expression as python code [WIP]

[kszucs]

depends on #4493

Closes https://github.com/ibis-project/ibis/issues/4335.

[github-actions[bot]]

Test Results

       39 files         39 suites   1h 28m 47s :stopwatch: 10 936 tests   8 538 :heavy_check_mark: 2 398 :zzz: 0 :x: 40 134 runs  30 848 :heavy_check_mark: 9 286 :zzz: 0 :x:

Results for commit 88ddf6cf.

:recycle: This comment has been updated with latest results.

[codecov[bot]]

Codecov Report

Merging #4535 (f9c6e70) into master (588df3f) will increase coverage by 0.04%. The diff coverage is 87.47%.

@@            Coverage Diff             @@
##           master    #4535      +/-   ##
==========================================
+ Coverage   92.34%   92.39%   +0.04%     
==========================================
  Files         183      183              
  Lines       20024    20111      +87     
  Branches     2966     2996      +30     
==========================================
+ Hits        18492    18582      +90     
- Misses       1151     1157       +6     
+ Partials      381      372       -9     

Impacted Files	Coverage Δ
ibis/backends/decompile/compiler.py	35.51% <35.51%> (ø)
ibis/backends/dask/execution/util.py	87.06% <80.00%> (ø)
ibis/common/graph.py	95.55% <95.55%> (ø)
ibis/expr/analysis.py	95.52% <95.65%> (-0.03%)	:arrow_down:
ibis/common/validators.py	99.06% <97.56%> (+4.28%)	:arrow_up:
ibis/backends/base/sql/__init__.py	89.10% <100.00%> (ø)
ibis/backends/base/sql/compiler/select_builder.py	97.32% <100.00%> (+0.01%)	:arrow_up:
ibis/backends/clickhouse/__init__.py	91.00% <100.00%> (-0.09%)	:arrow_down:
ibis/backends/impala/__init__.py	85.89% <100.00%> (ø)
ibis/backends/pandas/__init__.py	84.90% <100.00%> (-0.15%)	:arrow_down:
... and 31 more

[kszucs]

Going to split this PR into two, then address the review comments.

[kszucs]

I'd recommend marking this as an experimental feature since a couple of expressions don't roundtrip properly, most notably the reductions rewritten to aggregations.

On the other hand I managed to reach a rather high coverage using the already removed sqlgolden fixture to exercise decompile() on multiple expressions (see the temporarily committed, generated python files). @cpcloud could you recommend a method to properly test the decompiler without manually maintaining a bunch of tests?

I'm not planning to invest much more time to this PR other than finalizing it.

cc @saulpw @jcrist

[cpcloud]

@kszucs Can you use pytest-snapshot for the tests here? It's hard to see if you are because the large number of files changed makes the browser slow 😬

[kszucs]

@cpcloud updated, could you take another look?

[codecov[bot]]

Codecov Report

Merging #4535 (0154583) into master (0fabe8b) will decrease coverage by 0.65%. The diff coverage is 94.59%.

:exclamation: Current head 0154583 differs from pull request most recent head 88ddf6c. Consider uploading reports for the commit 88ddf6c to get more accurate results

@@            Coverage Diff             @@
##           master    #4535      +/-   ##
==========================================
- Coverage   92.87%   92.22%   -0.66%     
==========================================
  Files         189      190       +1     
  Lines       21459    21680     +221     
  Branches     2978     3008      +30     
==========================================
+ Hits        19931    19995      +64     
- Misses       1120     1275     +155     
- Partials      408      410       +2     

Impacted Files	Coverage Δ
ibis/expr/decompile.py	94.57% <94.57%> (ø)
ibis/expr/api.py	96.72% <100.00%> (ø)
ibis/expr/typing.py	0.00% <0.00%> (-100.00%)	:arrow_down:
ibis/backends/snowflake/registry.py	0.00% <0.00%> (-93.94%)	:arrow_down:
ibis/backends/snowflake/datatypes.py	0.00% <0.00%> (-89.40%)	:arrow_down:
ibis/backends/snowflake/__init__.py	4.61% <0.00%> (-84.62%)	:arrow_down:
ibis/backends/base/sql/alchemy/__init__.py	90.30% <0.00%> (-0.94%)	:arrow_down:
ibis/backends/dask/execution/util.py	86.95% <0.00%> (-0.12%)	:arrow_down:
ibis/backends/base/sql/__init__.py	93.38% <0.00%> (-0.06%)	:arrow_down:
ibis/expr/types/core.py	93.91% <0.00%> (-0.06%)	:arrow_down:
... and 35 more

[kszucs]

@cpcloud addressed the review comments and also did some refactoring

[cpcloud]

Sweet, merging!