rules_pkg icon indicating copy to clipboard operation
rules_pkg copied to clipboard

Fix pkg_tar directory entries

Open jameshilliard opened this issue 2 years ago • 6 comments

Directories appear to work for zip files but not tar files.

This should ensure we don't try to process directories as normal files.

Fixes issues like:

rules_pkg/pkg/private/tar/tar_writer.py", line 242, in add_file
    with open(file_content, 'rb') as f:
IsADirectoryError: [Errno 21] Is a directory: 'bazel-out/darwin-fastbuild/bin/py/selenium/webdriver/common/devtools/v106'

jameshilliard avatar Dec 07 '22 05:12 jameshilliard

This PR works well, I've got it in the Selenium tree for now.

AutomatedTester avatar Dec 07 '22 13:12 AutomatedTester

Could you provide test cases for this?

If this is supporting what I think it is (passing in directory labels that are not TreeArtifacts), I am hesitant to support this any further, as is not a best practice. @aiuto may have a differing opinion.

Please correct me if I'm wrong, but the relevant code in the Selenium tree is a genrule that looks to be emitting a directory, which is unfortunately unsound (the documentation's words, not mine) and IIRC Bazel emits a warning saying something to this point. This IMO should instead be a first-class rule that emits a TreeArtifact (via ctx.actions.declare_directory, if you didn't already know).

nacl avatar Dec 07 '22 16:12 nacl

Could you provide test cases for this?

Hmm, is there a similar case that I should base one off of?

If this is supporting what I think it is (passing in directory labels that are not TreeArtifacts), I am hesitant to support this any further, as is not a best practice. @aiuto may have a differing opinion.

Well...it's apparently supported for zip files already so the inconsistency with tar files is problematic.

Please correct me if I'm wrong, but the relevant code in the Selenium tree is a genrule that looks to be emitting a directory, which is unfortunately unsound (the documentation's words, not mine) and IIRC Bazel emits a warning saying something to this point. This IMO should instead be a first-class rule that emits a TreeArtifact (via ctx.actions.declare_directory, if you didn't already know).

Yeah, I think that probably also needs to be changed, although I'm not very familiar with bazel in general so making tar work like zip files seemed easier for a short term fix.

jameshilliard avatar Dec 07 '22 16:12 jameshilliard

I worry about this too. It looks like it will pull in trees that might not have been staged, so it may do what you want for local builds but not remote builds. If zip does that, it is probably an accidental side effect. Both support tree artifacts as inputs.

As far as tests, there already are tests for tree artifacts in tests/tar/BUILD.

aiuto avatar Dec 07 '22 16:12 aiuto

As far as tests, there already are tests for tree artifacts in tests/tar/BUILD.

Test which reproduces the error is added.

jameshilliard avatar Dec 08 '22 04:12 jameshilliard

Maybe, but that case should be working already.

aiuto avatar Dec 08 '22 22:12 aiuto