beam icon indicating copy to clipboard operation
beam copied to clipboard

[BEAM-13590]Add support for python 3.10

Open AnandInguva opened this issue 3 years ago • 21 comments

Add Python 3.10 infrastructure and test suites

Fixes: https://github.com/apache/beam/issues/21971 Fixes: https://github.com/apache/beam/issues/21458 FIxes: https://github.com/apache/beam/issues/21671


Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

  • [ ] Choose reviewer(s) and mention them in a comment (R: @username).
  • [ ] Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
  • [ ] Update CHANGES.md with noteworthy changes.
  • [ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md

GitHub Actions Tests Status (on master branch)

Build python source distribution and wheels Python tests Java tests

See CI.md for more information about GitHub Actions CI.

AnandInguva avatar May 18 '22 03:05 AnandInguva

Can one of the admins verify this patch?

asf-ci avatar May 18 '22 03:05 asf-ci

Can one of the admins verify this patch?

asf-ci avatar May 18 '22 03:05 asf-ci

Can one of the admins verify this patch?

asf-ci avatar May 18 '22 03:05 asf-ci

Codecov Report

Merging #17700 (e7504b4) into master (3ffdf8d) will decrease coverage by 1.06%. The diff coverage is 83.33%.

@@            Coverage Diff             @@
##           master   #17700      +/-   ##
==========================================
- Coverage   73.40%   72.34%   -1.07%     
==========================================
  Files         719      719              
  Lines       95940    95933       -7     
==========================================
- Hits        70424    69400    -1024     
- Misses      24205    25222    +1017     
  Partials     1311     1311              
Flag Coverage Δ
python 81.61% <83.33%> (-1.52%) :arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...ython/apache_beam/runners/interactive/sql/utils.py 76.09% <ø> (ø)
sdks/python/apache_beam/typehints/row_type.py 100.00% <ø> (ø)
...apache_beam/typehints/native_type_compatibility.py 85.52% <33.33%> (-1.06%) :arrow_down:
sdks/python/apache_beam/typehints/opcodes.py 85.35% <50.00%> (-0.26%) :arrow_down:
.../python/apache_beam/typehints/trivial_inference.py 96.15% <87.50%> (-0.27%) :arrow_down:
sdks/python/apache_beam/__init__.py 85.00% <100.00%> (ø)
...apache_beam/runners/dataflow/internal/apiclient.py 77.34% <100.00%> (ø)
...s/python/apache_beam/typehints/sharded_key_type.py 93.54% <100.00%> (ø)
sdks/python/apache_beam/typehints/typehints.py 93.37% <100.00%> (-0.06%) :arrow_down:
...i/org/apache/beam/model/pipeline/v1/metrics_pb2.py 21.08% <0.00%> (-78.92%) :arrow_down:
... and 16 more

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

codecov[bot] avatar May 18 '22 03:05 codecov[bot]

Run Python PreCommit

AnandInguva avatar Jun 01 '22 17:06 AnandInguva

Run Python PreCommit

AnandInguva avatar Jun 01 '22 17:06 AnandInguva

What is the next step on this PR?

aaltay avatar Jun 09 '22 20:06 aaltay

What is the next step on this PR?

There are type hint related bugs. I will do more investigation next week and try to find the solution or ask help.

AnandInguva avatar Jun 09 '22 21:06 AnandInguva

Is this still being worked on? Are there intermediate milestones that could be checked in sooner rather than later?

robertwb avatar Jul 15 '22 19:07 robertwb

Is this still being worked on? Are there intermediate milestones that could be checked in sooner rather than later?

Yes, it is still being worked on. There are some blockers and we filed GH issues on them.

AnandInguva avatar Jul 16 '22 04:07 AnandInguva

Run Python 3.10 PostCommit

AnandInguva avatar Jul 27 '22 17:07 AnandInguva

Run Python 3.10 PostCommit

AnandInguva avatar Jul 27 '22 18:07 AnandInguva

Run Python PreCommit

AnandInguva avatar Sep 05 '22 03:09 AnandInguva

Just checking: Is this PR blocked? What are the next steps?

aaltay avatar Sep 15 '22 01:09 aaltay

Just checking: Is this PR blocked? What are the next steps?

Waiting on Shuffle client for Runner v1. Planning to include this PR before next release cut.

AnandInguva avatar Sep 15 '22 02:09 AnandInguva

Run Python Dataflow V2 ValidatesRunner

AnandInguva avatar Sep 19 '22 20:09 AnandInguva

Run Python 3.10 PostCommit

AnandInguva avatar Sep 19 '22 20:09 AnandInguva

Run Python Examples_Direct

AnandInguva avatar Sep 19 '22 20:09 AnandInguva

Run Python Dataflow ValidatesContainer

AnandInguva avatar Sep 19 '22 20:09 AnandInguva

Run Python Dataflow ValidatesContainer

AnandInguva avatar Sep 19 '22 22:09 AnandInguva

Run Python 3.10 PostCommit

AnandInguva avatar Sep 21 '22 03:09 AnandInguva

@tvalentyn Can you enable the gh workflow?

AnandInguva avatar Oct 09 '22 02:10 AnandInguva

Run Python Dataflow ValidatesRunner

AnandInguva avatar Oct 09 '22 15:10 AnandInguva

Run Python Dataflow ValidatesContainer

AnandInguva avatar Oct 09 '22 15:10 AnandInguva

Run Python Dataflow ValidatesRunner

AnandInguva avatar Oct 09 '22 15:10 AnandInguva

Run Python Dataflow V2 ValidatesRunner

AnandInguva avatar Oct 09 '22 15:10 AnandInguva

Run Python Examples_Direct

AnandInguva avatar Oct 09 '22 15:10 AnandInguva

Run Python 3.10 PostCommit

AnandInguva avatar Oct 10 '22 19:10 AnandInguva

@tvalentyn Can you enable the gh workflow?

I clicked the button to approve the run, lmk if other action is necessary.

tvalentyn avatar Oct 10 '22 19:10 tvalentyn

Was this merged?

aaltay avatar Oct 20 '22 23:10 aaltay