beam
beam copied to clipboard
Mltransform Criteo 10GB perf test
Please add a meaningful description for your change here
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
- [ ] Mention the appropriate issue in your description (for example:
addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, commentfixes #<ISSUE NUMBER>instead. - [ ] Update
CHANGES.mdwith noteworthy changes. - [ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.
See the Contributor Guide for more tips on how to make review process smoother.
To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md
GitHub Actions Tests Status (on master branch)
See CI.md for more information about GitHub Actions CI or the workflows README to see a list of phrases to trigger workflows.
https://github.com/apache/beam/actions/runs/7191759283/job/19604271438 - successful run.
MLTransform - https://pantheon.corp.google.com/dataflow/jobs/us-central1/2023-12-13_07_32_17-4151282941601198289;step=ref_AppliedPTransform_Map-lambda-at-criteo-py-98-_11;graphView=0?project=apache-beam-testing&pageState=(%22dfTime%22:(%22l%22:%22dfJobMaxTime%22))&e=13802955&mods=-ai_platform_fake_service,ai_platform_staging_service
TFT - https://pantheon.corp.google.com/dataflow/jobs/us-central1/2023-12-12_23_03_42-286055797994519897;step=DecodeForTransform%2FRawRecordBeamSource;mainTab=JOB_GRAPH;graphView=0?project=apache-beam-testing&pageState=(%22dfTime%22:(%22l%22:%22dfJobMaxTime%22))&e=13802955&mods=-ai_platform_fake_service,ai_platform_staging_service
Both are not apples to apples comparison but we can gain some insights and any potential improvements. I will see if batching elements works.
This pull request has been marked as stale due to 60 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the [email protected] list. Thank you for your contributions.
This pull request has been closed due to lack of activity. If you think that is incorrect, or the pull request requires review, you can revive the PR at any time.