Race condition when depending on rules created in post_build.

[cemeceme]

Here is an example of a BUILD file with a post_build function adding dependencies:

def post_func(ruleName, cmdOutput):
    fileRule = text_file(name="postFile", content = "Some content", out="postFile.txt")
    add_dep(ruleName, fileRule, True)

genrule(name="postRule", cmd="", post_build=post_func)

genrule(name="postRuleConsumer", cmd="", deps=[":postRule"])

Running this with plz build //:postRuleConsumer will result in an output of cannot calculate hash for plz-out/gen/postFile.txt: file does not exist.

However, if you somehow force post_func to run without dependents (in this case postRuleConsumer), for example by calling //:postRule on its own, then postFile.txt will get created. Once the file is created, subsequent runs will start to succeed and have the file available until the next plz clean call.

What seems to be happening, is that the commands of rules in post_builds aren't getting executed. I have confirmed this by changing the text_file rule into genrule(name="postFile", cmd = "sleep 10; touch postFile.txt", out="postFile.txt"). When //:postRule is run on its own, it will take 10 seconds to run, however when run with //:postRuleConsumer the command will immediately finish, either with the same error, or by succeeding if //:postRule was run on its own beforehand.

[cemeceme]

I have been looking into the code base to track down this issue, and found that it was actually a race condition. It turns out that if there is another rule that can block the dependent rule until the post build completes, then everything works as expected. Otherwise, the same issue as mentioned occurs.

Here is an example build script that will succeed:

def post_func(ruleName, cmdOutput):
    fileRule = genrule(name="postFile", cmd = "sleep 10; touch postFile.txt", out="postFile.txt")
    add_dep(ruleName, fileRule, True)

genrule(name="postRule", cmd="", post_build=post_func)

genrule(name="waitRule", cmd="sleep 11; touch waitFile.txt", out="waitFile.txt")

genrule(name="postRuleConsumer", cmd="", deps=[":postRule", ":waitRule"])

Note the sleep 10 in the file rule and sleep 11 in the wait rule. As long as the "postFile" rule in this example completes before the "waitRule" everything works.