bazelbuild
`pip.parse` should allow hiding transitive dependencies
🚀 feature request
Relevant Rules
pip.parse
Description
Given a requirements.in file containing only foo==4.2.0 from which I create a lock file with content
foo==4.2.0 --hash=<some_hash>
dep_of_foo==1.33.7 --hash=<some_hash>
When using pip.parse to crate a hub @pypi from this lock file users can access all Python modules from the lock file. Concretely, @pypi//foo and @pypi//dep_of_foo.
I consider dep_of_foo an implementation detail which no user should depend on. When changing the version of foo, then dep_of_foo might vanish or change drastically as side effect. If I wanted users to access dep_of_foo, I would have added it to the requirements.in file to make it an explicit and desired direct dependency of my project.
It would be great if there were an option enforcing that transitive dependencies are not available to users.
Describe the solution you'd like
Ideally pip.parse would offer an attribute restrict_visibility_to (or any other name) which takes a file list. Then, one could provide one or multiple requirements.in files to this attribute. pip.parse can then read those files, extract the Python module names and ensure only those are public targets in the pip hub.
The implementation would be easier if restrict_visibility_to takes a list of strings and the user explicitly states which Python modules should be public. However, I consider this inferior, as it increases the maintenance burden whenever changing the requirements.in files.
Describe alternatives you've considered
I implemented the described behavior locally as a workspace rule, which creates a new hub with alias targets pointing to the hub created by pip.parse. It is trivial to do so, not much logic is required.
However, this means one has to teach people not to use the original hub created by pip.parse. Or one has to write yet another piece of custom code for a BUILD file checker ensuring this rule.
Overall, it would be much nicer if this behavior would be a feature of upstream pip.parse.
Currently we have this is_exposed flag internally in the bzlmod extension. The idea of reading the requirements.in files for this information is interesting and should work as our requirements_txt parser should handle those. It may be that we would need to implement a lightweight pyproject.toml parser for the cases where we are using pyproject.toml files instead.
I can mentor anyone who wants to take a stab at it. Doing it first for requirements.in support only is fine, IMHO. Users can use them even if they are using pyproject.toml files.
I believe rules_jvm_external offers this. It could be nice if the semantics/interface were similar since I assume that it is relatively battle tested, and also it's always nice when rulesets have components that feel relatively generic between them.
