kedro
kedro copied to clipboard
Rename `_transcoding` module, deprecate old constant
Description
This leaves TRANSCODING_SEPARATOR
in a public module; there is no expectation that it would move to a private module.
Discussion in #3826.
Developer notes
Kedro-Viz will be fine, but https://github.com/kedro-org/kedro-viz/blob/main/package/kedro_viz/integrations/kedro/hooks.py#L15-L17 could get dropped again, if so desired. (Nobody needs to be on Kedro 0.19.4, since there will be no functional differences between that and 0.19.5.)
Alternatives considered
It's possible to rename the new _transcoding
module to transcoding
, but this avoids that and goes back to something very similar to what was there in 0.19.3 and below.
Developer Certificate of Origin
We need all contributions to comply with the Developer Certificate of Origin (DCO). All commits must be signed off by including a Signed-off-by
line in the commit message. See our wiki for guidance.
If your PR is blocked due to unsigned commits, then you must follow the instructions under "Rebase the branch" on the GitHub Checks page for your PR. This will retroactively add the sign-off to all unsigned commits and allow the DCO check to pass.
Checklist
- [x] Read the contributing guidelines
- [x] Signed off each commit with a Developer Certificate of Origin (DCO)
- [ ] Opened this PR as a 'Draft Pull Request' if it is work-in-progress
- [ ] Updated the documentation to reflect the code changes
- [ ] Added a description of this change in the
RELEASE.md
file - [ ] Added tests to cover my changes
- [ ] Checked if this change will affect Kedro-Viz, and if so, communicated that with the Viz team
I do think that the private module is leaky, the methods are private but then we expose TRANSCODING_SEPARATOR
to the public module.
The only thing absolutely blocking here is the circular dependency, this PR is an alternative to it. @deepyaman make a point that it shouldn't be just 0.19 backward compatible, would we should continue to expose TRASNCODING_SEPARATOR
for 0.20. I think this convince me a bit more towards moving it to a public module.
For private/public, there are different ways.
- Move the module to public as
transcoding.py
instead of_transcoding.py
, hide the unnecessary thing with__all__
. (I don't think it's needed because the rest of the method are private already)
I agree with the point of moving TRANSCODING_SEPARATOR
to public, but since _strip_transcoding
is also imported and used by other components, it should be public as well. So, the solution of moving from _transcoding.py
to transcoding.py
makes the most sense to me.
IMO, from the implementational point of view, keeping transcoding logic in a separate file looks better rather than moving it back to the pipeline and fixing circular dependency.
I agree with the point of moving
TRANSCODING_SEPARATOR
to public, but since_strip_transcoding
is also imported and used by other components, it should be public as well. So, the solution of moving from_transcoding.py
totranscoding.py
makes the most sense to me.IMO, from the implementational point of view, keeping transcoding logic in a separate file looks better rather than moving it back to the pipeline and fixing circular dependency.
Since there is some time to discuss this now—what will be the recommended/supported way to import TRANSCODING_SEPARATOR
(and maybe strip_transcoding
) going forward? Will it be:
-
from kedro.pipeline.transcoding import TRANSCODING_SEPARATOR
-
from kedro.pipeline.pipeline import TRANSCODING_SEPARATOR
(I assume this won't be recommended, but code already uses it; will this be deprecated?) - Both?
I agree with the point of moving
TRANSCODING_SEPARATOR
to public, but since_strip_transcoding
is also imported and used by other components, it should be public as well. So, the solution of moving from_transcoding.py
totranscoding.py
makes the most sense to me. IMO, from the implementational point of view, keeping transcoding logic in a separate file looks better rather than moving it back to the pipeline and fixing circular dependency.Since there is some time to discuss this now—what will be the recommended/supported way to import
TRANSCODING_SEPARATOR
(and maybestrip_transcoding
) going forward? Will it be:
from kedro.pipeline.transcoding import TRANSCODING_SEPARATOR
from kedro.pipeline.pipeline import TRANSCODING_SEPARATOR
(I assume this won't be recommended, but code already uses it; will this be deprecated?)- Both?
Do both for 0.19, keep 1 only starting from 0.20 going forward. Similar to https://github.com/kedro-org/kedro/pull/1837/files#diff-c9b9e2fdad60057c915a16d9caf8c11637750cd6094585b4ad2f583df619ddac
This helps to avoid codebase diverge from main
, we only need to remove the alias in 0.20
I agree with the point of moving
TRANSCODING_SEPARATOR
to public, but since_strip_transcoding
is also imported and used by other components, it should be public as well. So, the solution of moving from_transcoding.py
totranscoding.py
makes the most sense to me. IMO, from the implementational point of view, keeping transcoding logic in a separate file looks better rather than moving it back to the pipeline and fixing circular dependency.Since there is some time to discuss this now—what will be the recommended/supported way to import
TRANSCODING_SEPARATOR
(and maybestrip_transcoding
) going forward? Will it be:
from kedro.pipeline.transcoding import TRANSCODING_SEPARATOR
from kedro.pipeline.pipeline import TRANSCODING_SEPARATOR
(I assume this won't be recommended, but code already uses it; will this be deprecated?)- Both?
Do both for 0.19, keep 1 only starting from 0.20 going forward. Similar to https://github.com/kedro-org/kedro/pull/1837/files#diff-c9b9e2fdad60057c915a16d9caf8c11637750cd6094585b4ad2f583df619ddac
This helps to avoid codebase diverge from
main
, we only need to remove the alias in 0.20
Done!
However, we might consider exposing methods to apply the transcoding instead of the constant. It will guarantee that users follow the same logic when applying it rather than implementing their own based on the constant.
FWIW I actually have found access to the constant most useful in plugin development. Maybe functions to apply transcoding, split on separator, and check if something is transcoded could work, but I think it's less a user need in my eyes.