Repository navigation
Propagate stable code identifier pickling and preserve pickler config - #40453
kristynsmith wants to merge 4 commits into
Conversation
Replace config assignment with dataclasses.replace for stability.
|
Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #40453 +/- ##
=============================================
+ Coverage 56.20% 79.56% +23.35%
=============================================
Files 1123 487 -636
Lines 178213 82226 -95987
Branches 1489 0 -1489
=============================================
- Hits 100168 65421 -34747
+ Misses 75524 16805 -58719
+ Partials 2521 0 -2521
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
Assigning reviewers: R: @tvalentyn for label python. This pull request likely touches a core component ("core" label). Please review with scrutiny. Note: If you would like to opt out of this review, comment Available commands:
The PR bot will only process comments in the main thread (not review comments). |
|
Reached out to @kristynsmith offline |
|
waiting on author |
Previously,
Pipeline.to_runner_apidid not forward the runner'senable_stable_code_identifier_picklingsetting ontoPipelineContext, preventing downstream transforms from using stable code object identifiers during Runner API serialization. Additionally, enablingenable_stable_code_identifier_picklingincloudpickle_picklerreplaced the active config with a staticSTABLE_CODE_IDENTIFIER_CONFIGinstance initialized at import time, discarding runtime overrides made toDEFAULT_CONFIG(such as customid_generatorandpickle_main_by_refsettings).Copy
enable_stable_code_identifier_picklingfrom the runner ontoPipelineContextinPipeline.to_runner_api, and usedataclasses.replaceincloudpickle_pickler._dumpssoget_code_object_paramsis updated without overwriting other fields on the activeCloudPickleConfig.Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, commentfixes #<ISSUE NUMBER>instead.CHANGES.mdwith noteworthy changes.See the Contributor Guide for more tips on how to make review process smoother.
To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md
GitHub Actions Tests Status (on master branch)
See CI.md for more information about GitHub Actions CI or the workflows README for a list of workflows and how to trigger them.