kyuubi icon indicating copy to clipboard operation
kyuubi copied to clipboard

[KYUUBI #3773][FLINK] Support Flink Python UDF

Open a49a opened this issue 3 years ago • 5 comments

Why are the changes needed?

Python support version of Flink matrix: Flink 1.14: Python 3.6, 3.7 or 3.8 Flink 1.15: Python 3.6, 3.7 or 3.8 Flink 1.16: Python 3.6, 3.7, 3.8 or 3.9

How was this patch tested?

  • [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

  • [ ] Add screenshots for manual tests if appropriate

  • [X] Run test locally before make a pull request

a49a avatar Nov 08 '22 03:11 a49a

Codecov Report

Merging #3779 (1142665) into master (7644849) will decrease coverage by 0.13%. The diff coverage is 100.00%.

:exclamation: Current head 1142665 differs from pull request most recent head 6e8541b. Consider uploading reports for the commit 6e8541b to get more accurate results

@@             Coverage Diff              @@
##             master    #3779      +/-   ##
============================================
- Coverage     53.31%   53.19%   -0.13%     
  Complexity       13       13              
============================================
  Files           577      569       -8     
  Lines         31584    31060     -524     
  Branches       4248     4194      -54     
============================================
- Hits          16838    16521     -317     
+ Misses        13156    12985     -171     
+ Partials       1590     1554      -36     
Impacted Files Coverage Δ
...ache/kyuubi/engine/flink/FlinkProcessBuilder.scala 73.52% <100.00%> (+2.10%) :arrow_up:

... and 71 files with indirect coverage changes

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

codecov-commenter avatar Feb 21 '23 09:02 codecov-commenter

What's the Flink Python version support matrix? Do you suppose to let Kyuubi fully match?

pan3793 avatar Feb 21 '23 09:02 pan3793

What's the Flink Python version support matrix? Do you suppose to let Kyuubi fully match?

The Python version of Flink is unrelated to Kyuubi. Users can refer to Flink Python version support and set the python.executable option to specify the Python interpreter version.

So Kyuubi Flink SQL engine can ignore the Flink Python version problem.

a49a avatar Feb 21 '23 10:02 a49a

I have checked out CI failed. I need to install the pyflink module in Python of the CI environment.

a49a avatar Feb 22 '23 02:02 a49a

Kyuubi 1.9 goes to support Flink 1.16 to 1.19, would you like to rebase this PR and switch to python 3.9?

pan3793 avatar Oct 27 '23 04:10 pan3793