kedro icon indicating copy to clipboard operation
kedro copied to clipboard

Rename `_transcoding` module, deprecate old constant

Open deepyaman opened this issue 10 months ago • 6 comments

Description

This leaves TRANSCODING_SEPARATOR in a public module; there is no expectation that it would move to a private module. Discussion in #3826.

Developer notes

Kedro-Viz will be fine, but https://github.com/kedro-org/kedro-viz/blob/main/package/kedro_viz/integrations/kedro/hooks.py#L15-L17 could get dropped again, if so desired. (Nobody needs to be on Kedro 0.19.4, since there will be no functional differences between that and 0.19.5.)

Alternatives considered

It's possible to rename the new _transcoding module to transcoding, but this avoids that and goes back to something very similar to what was there in 0.19.3 and below.

Developer Certificate of Origin

We need all contributions to comply with the Developer Certificate of Origin (DCO). All commits must be signed off by including a Signed-off-by line in the commit message. See our wiki for guidance.

If your PR is blocked due to unsigned commits, then you must follow the instructions under "Rebase the branch" on the GitHub Checks page for your PR. This will retroactively add the sign-off to all unsigned commits and allow the DCO check to pass.

Checklist

  • [x] Read the contributing guidelines
  • [x] Signed off each commit with a Developer Certificate of Origin (DCO)
  • [ ] Opened this PR as a 'Draft Pull Request' if it is work-in-progress
  • [ ] Updated the documentation to reflect the code changes
  • [ ] Added a description of this change in the RELEASE.md file
  • [ ] Added tests to cover my changes
  • [ ] Checked if this change will affect Kedro-Viz, and if so, communicated that with the Viz team

deepyaman avatar Apr 19 '24 16:04 deepyaman

I do think that the private module is leaky, the methods are private but then we expose TRANSCODING_SEPARATOR to the public module.

The only thing absolutely blocking here is the circular dependency, this PR is an alternative to it. @deepyaman make a point that it shouldn't be just 0.19 backward compatible, would we should continue to expose TRASNCODING_SEPARATOR for 0.20. I think this convince me a bit more towards moving it to a public module.

For private/public, there are different ways.

  1. Move the module to public as transcoding.py instead of _transcoding.py, hide the unnecessary thing with __all__. (I don't think it's needed because the rest of the method are private already)

noklam avatar Apr 22 '24 10:04 noklam

I agree with the point of moving TRANSCODING_SEPARATOR to public, but since _strip_transcoding is also imported and used by other components, it should be public as well. So, the solution of moving from _transcoding.py to transcoding.py makes the most sense to me.

IMO, from the implementational point of view, keeping transcoding logic in a separate file looks better rather than moving it back to the pipeline and fixing circular dependency.

ElenaKhaustova avatar Apr 22 '24 11:04 ElenaKhaustova

I agree with the point of moving TRANSCODING_SEPARATOR to public, but since _strip_transcoding is also imported and used by other components, it should be public as well. So, the solution of moving from _transcoding.py to transcoding.py makes the most sense to me.

IMO, from the implementational point of view, keeping transcoding logic in a separate file looks better rather than moving it back to the pipeline and fixing circular dependency.

Since there is some time to discuss this now—what will be the recommended/supported way to import TRANSCODING_SEPARATOR (and maybe strip_transcoding) going forward? Will it be:

  • from kedro.pipeline.transcoding import TRANSCODING_SEPARATOR
  • from kedro.pipeline.pipeline import TRANSCODING_SEPARATOR (I assume this won't be recommended, but code already uses it; will this be deprecated?)
  • Both?

deepyaman avatar Apr 22 '24 15:04 deepyaman

I agree with the point of moving TRANSCODING_SEPARATOR to public, but since _strip_transcoding is also imported and used by other components, it should be public as well. So, the solution of moving from _transcoding.py to transcoding.py makes the most sense to me. IMO, from the implementational point of view, keeping transcoding logic in a separate file looks better rather than moving it back to the pipeline and fixing circular dependency.

Since there is some time to discuss this now—what will be the recommended/supported way to import TRANSCODING_SEPARATOR (and maybe strip_transcoding) going forward? Will it be:

  • from kedro.pipeline.transcoding import TRANSCODING_SEPARATOR
  • from kedro.pipeline.pipeline import TRANSCODING_SEPARATOR (I assume this won't be recommended, but code already uses it; will this be deprecated?)
  • Both?

Do both for 0.19, keep 1 only starting from 0.20 going forward. Similar to https://github.com/kedro-org/kedro/pull/1837/files#diff-c9b9e2fdad60057c915a16d9caf8c11637750cd6094585b4ad2f583df619ddac

This helps to avoid codebase diverge from main, we only need to remove the alias in 0.20

noklam avatar Apr 22 '24 15:04 noklam

I agree with the point of moving TRANSCODING_SEPARATOR to public, but since _strip_transcoding is also imported and used by other components, it should be public as well. So, the solution of moving from _transcoding.py to transcoding.py makes the most sense to me. IMO, from the implementational point of view, keeping transcoding logic in a separate file looks better rather than moving it back to the pipeline and fixing circular dependency.

Since there is some time to discuss this now—what will be the recommended/supported way to import TRANSCODING_SEPARATOR (and maybe strip_transcoding) going forward? Will it be:

  • from kedro.pipeline.transcoding import TRANSCODING_SEPARATOR
  • from kedro.pipeline.pipeline import TRANSCODING_SEPARATOR (I assume this won't be recommended, but code already uses it; will this be deprecated?)
  • Both?

Do both for 0.19, keep 1 only starting from 0.20 going forward. Similar to https://github.com/kedro-org/kedro/pull/1837/files#diff-c9b9e2fdad60057c915a16d9caf8c11637750cd6094585b4ad2f583df619ddac

This helps to avoid codebase diverge from main, we only need to remove the alias in 0.20

Done!

deepyaman avatar May 09 '24 05:05 deepyaman

However, we might consider exposing methods to apply the transcoding instead of the constant. It will guarantee that users follow the same logic when applying it rather than implementing their own based on the constant.

FWIW I actually have found access to the constant most useful in plugin development. Maybe functions to apply transcoding, split on separator, and check if something is transcoded could work, but I think it's less a user need in my eyes.

deepyaman avatar May 14 '24 14:05 deepyaman