facebook
How can I use aquery in bxl to get all actions for all targets?
I am trying to iterate over all configured targets, then enumerate each of their actions. aquery requires a target literal, so I can't seem to just do aquery(...). So I tried to enumerate all configured_targets, and then pass each one to aquery. First I ran into the problem that aquery seems to want an unconfigured target label.
for target in ctx.configured_targets('...'):
aquery_result = ctx.aquery(target.label)
> error: Type of parameter `target_platform` doesn't match, expected `None | str | target_label`, actual `configured_target_label`
So I changed that to target.label.raw_target().
That worked and gave me something that I could call the function all_actions() on, but it seems that function requires me to pass in another target. Which I'm confused about, since I had to pass in a target to ctx.aquery in the first place. Example:
for target in ctx.configured_targets('...'):
aquery_result = ctx.aquery(target.label.raw_target())
all_actions = aquery_result.all_actions()
> error: Missing parameter `targets` for call to all_actions
Not entirely sure what I'm supposed to do at this point, so I tried to pass in the configured target label again, but then it always returns an empty list of actions, even when it's clear the target does have actions on it.
I tried to follow this documentation here (https://buck2.build/docs/rfcs/bxl/#actions) which has the following example:
targets = deps(“foo:rule”)
for t in targets:
action_ctx = ctx.analysis(t).actions
for action in action_ctx.iter():
if “foo/path” in action.output:
ctx.build(action)
by modifying it to the following, which gives me a different error
for t in ctx.configured_targets("..."):
action_ctx = ctx.analysis(t).actions
> error: Object of type `analysis_result` has no attribute `actions`
Any advice about how to proceed?
The parameter that ctx.aquery() takes is actually an optional target platform label, and not the query targets. The target platform would be used to configure any targets passed into any aquery queries, such as eval, all_actions, or all_outputs, and if none is provided, we configure the targets with their default target platform.
You're almost there - it should be something like this:
targets = ctx.configured_targets('...')
aquery_ctx = ctx.aquery() # creates an aquery context
result = aquery_ctx.all_actions(targets) # you can pass in the entire target set into the query
# Alternatively, you don't necessarily need to configure the targets yourself first, and just pass in the string literal target expression directly
result = ctx.aquery().all_actions("...")
for node in result:
# inspect the action nodes
Action node API docs can be found here.
The doc with the analysis example you linked is an RFC, so the API has changed considerably. Indeed, analysis_result does not have an actions attribute, and we don't have plans to implement that.
I'll add a header to the RFC saying the APIs are probably outdated... or maybe get rid of it altogether to prevent confusion. I'll also add a section in the docs about querying in general.
Actually, re-opening this because it seems I'm still not quite there yet. For some reason, I'm getting duplicate entries for certain cpp files: a pic version and a non pic version. Here's what I get for a certain target when I run my bxl. I'm cutting a lot of the output on purpose in the interest of saving space.
D:\src\foo>buck2 bxl root//build-scripts/iterate_actions.bxl:iterate_actions
Build ID: b24ce1e7-a503-4084-ae7f-04bdfd1abb8e
Network: Up: 0B Down: 0B
Jobs completed: 1. Time elapsed: 0.0s.
<snip>
(target: `root//foo:bar (foo//platforms:my-platform#23b3e4bdecafe14a)`, id: `6`)
<snip>
(target: `root//foo:bar (foo//platforms:my-platform#23b3e4bdecafe14a)`, id: `12`)
<snip>
The important thing here is that there is an action with id 6 and another with id 12. If I instead run buck2 aquery "deps(root//foo:bar") on this target from the command line:
(target: `root//foo:bar (foo//platforms:my-platform#23b3e4bdecafe14a)`, id: `1`)
(target: `root//foo:bar (foo//platforms:my-platform#23b3e4bdecafe14a)`, id: `4`)
(target: `root//foo:bar (foo//platforms:my-platform#23b3e4bdecafe14a)`, id: `17`)
(target: `root//foo:bar (foo//platforms:my-platform#23b3e4bdecafe14a)`, id: `16`)
(target: `root//foo:bar (foo//platforms:my-platform#23b3e4bdecafe14a)`, id: `15`)
(target: `root//foo:bar (foo//platforms:my-platform#23b3e4bdecafe14a)`, id: `14`)
(target: `root//foo:bar (foo//platforms:my-platform#23b3e4bdecafe14a)`, id: `13`)
(target: `root//foo:bar (foo//platforms:my-platform#23b3e4bdecafe14a)`, id: `12`)
(target: `root//foo:bar (foo//platforms:my-platform#23b3e4bdecafe14a)`, id: `43`)
(target: `root//foo:bar (foo//platforms:my-platform#23b3e4bdecafe14a)`, id: `44`)
In this case I haven't snipped any output related to the target foo:bar. The point here is that there is an id of 12, but not an id of 6. So buck2 aquery (command line) is finding a reduced set of actions as my bxl, and I'm not sure why. Here is what actions 6 and actions 12 look like, when I print out their attrs in my bxl.
(target: `root//foo:bar (foo//platforms:my-platform#23b3e4bdecafe14a)`, id: `6`)
struct(
kind=run,
category=cxx_compile,
identifier=CLI/Repl.cpp (pic),
inputs=,
outputs=,
cmd=[
deps/msvc-hermetic/msvc/bin/Hostx64/x64/cl.exe,
/Fobuck-out\v2\gen\root\23b3e4bdecafe14a\Client\Luau\__cli__\__objects__\CLI\Repl.cpp.pic.obj,
@buck-out\v2\gen\root\23b3e4bdecafe14a\Client\Luau\__cli__\.cpp.argsfile,
-c,
Client\Luau\CLI\Repl.cpp
],
executor_preference=Default,
always_print_stderr=false,
weight=1,
dep_files=[],
metadata_param=None,
no_outputs_cleanup=false,
allow_cache_upload=false,
allow_dep_file_cache_upload=false,
executor_configuration=Local + use persistent workers false
)
(target: `root//foo:bar (foo//platforms:my-platform#23b3e4bdecafe14a)`, id: `12`)
struct(
kind=run,
category=cxx_compile,
identifier=CLI/Repl.cpp,
inputs=,
outputs=,
cmd=[
deps/msvc-hermetic/msvc/bin/Hostx64/x64/cl.exe,
/Fobuck-out\v2\gen\root\23b3e4bdecafe14a\Client\Luau\__cli__\__objects__\CLI\Repl.cpp.obj,
@buck-out\v2\gen\root\23b3e4bdecafe14a\Client\Luau\__cli__\.cpp.argsfile,
-c,
Client\Luau\CLI\Repl.cpp
],
executor_preference=Default,
always_print_stderr=false,
weight=1,
dep_files=[],
metadata_param=None,
no_outputs_cleanup=false,
allow_cache_upload=false,
allow_dep_file_cache_upload=false,
executor_configuration=Local + use persistent workers false
)
The only difference is the presence of (pic) in the filename. Which is strange, because there's nothing particularly interesting or unique about the way I define this target, certainly I don't specify anything having to do with pic in its rule invocation.
And for completeness, here is what my bxl looks like (again, snipped to only show the inner loop that iterates the actions):
def iterate_actions_impl(ctx):
targets = [ctx.configured_targets('root//foo:bar')]
aquery = ctx.aquery()
for target in targets:
for action in target_actions:
ctx.output.print(action)
iterate_actions = bxl_main(
impl = iterate_actions_impl,
cli_args = {}
)
For some reason, I'm getting duplicate entries for certain cpp files: a pic version and a non pic version
These aren't duplicate entries - these should be actually different actions based on if the pic flag is set or not. You can see the cmd_args is different between the entries.
certainly I don't specify anything having to do with pic in its rule invocation
Right, looks like the pic flag is set by the rules layer for you based on certain attributes/conditions of the target. For reference, the pic flag used in the cxx_compile implementation, and cxx_compile is used within cxx_library and cxx_executable.
Sanity check - are you also using deps query in your BXL, or are you using all_actions? The actual query wasn't included in your BXL snippet, but in the initial issue, you mentioned all_actions
But what I don't understand is that they're from the same target. For a given target platform (in this case the one with configuration hash 23b3e4bdecafe14a), why would the same target be compiled two different ways, with and without pic? I don't actually specify any pic related flags in my cxx_library either. The definition of this library is very simple and not really any different from any other libraries, where I'm not seeing these (pic) translation units. But when I aquery it in bxl, the same target is trying to compile the file two different ways, with and without pic. And when I aquery it from the command line, I don't see the pic version.
To answer your other question, I am not using deps query in bxl. I'm using all_actions. Is there a difference? TBH, I'm actually not sure why I have to use deps on the command line in the first place. My original intuition long time ago was that running buck2 aquery root//foo:bar would list all of the actions for that target. But it doesn't, it only lists one action of type run and category archive. So I just figured out through experimentation that if I use buck2 aquery deps(root//foo:bar) then I would get all of the actions. Example:
$ buck2 aquery root//foo:bar
Build ID: b5c9e67d-e945-45ac-97a2-21c9a7834bf8
Jobs completed: 1. Time elapsed: 0.0s.
(target: `root//foo:bar (root//platforms:my-platform#23b3e4bdecafe14a)`, id: `44`)
root//foo:bar (root//platforms:my-platform#23b3e4bdecafe14a)
$ buck2 aquery deps(root//foo:bar)
Build ID: 7201d529-e88d-46a6-854f-3913948549d9
Jobs completed: 1. Time elapsed: 0.0s.
root//foo:bar (root//platforms:my-platform#23b3e4bdecafe14a)
(target: `root//:a (root//platforms:my-platform#23b3e4bdecafe14a)`, id: `3`)
(target: `root//:b (root//platforms:my-platform#23b3e4bdecafe14a)`, id: `1`)
(target: `root//:c (root//platforms:my-platform#23b3e4bdecafe14a)`, id: `1`)
(target: `root//:d (root//platforms:my-platform#23b3e4bdecafe14a)`, id: `1`)
(target: `root//:e (root//platforms:my-platform#23b3e4bdecafe14a)`, id: `1`)
(target: `root//foo:bar (root//platforms:my-platform#23b3e4bdecafe14a)`, id: `1`)
(target: `root//foo:bar (root//platforms:my-platform#23b3e4bdecafe14a)`, id: `4`)
(target: `root//foo:bar (root//platforms:my-platform#23b3e4bdecafe14a)`, id: `17`)
(target: `root//foo:bar (root//platforms:my-platform#23b3e4bdecafe14a)`, id: `16`)
(target: `root//foo:bar (root//platforms:my-platform#23b3e4bdecafe14a)`, id: `15`)
(target: `root//foo:bar (root//platforms:my-platform#23b3e4bdecafe14a)`, id: `14`)
(target: `root//foo:bar (root//platforms:my-platform#23b3e4bdecafe14a)`, id: `13`)
(target: `root//foo:bar (root//platforms:my-platform#23b3e4bdecafe14a)`, id: `12`)
(target: `root//foo:bar (root//platforms:my-platform#23b3e4bdecafe14a)`, id: `43`)
(target: `root//foo:bar (root//platforms:my-platform#23b3e4bdecafe14a)`, id: `44`)
But now I have to do additional filtering on these because I didn't really want the deps (first 5 targets in this list) to begin with. All I wanted was the lines at the end (the ones that begin with target: root//foo:bar. Is this a bug in aquery or am I just misunderstanding the basic semantics?
For a given target platform (in this case the one with configuration hash 23b3e4bdecafe14a), why would the same target be compiled two different ways, with and without pic?
I think we would need the cxx rule authors to answer here, or maybe @cjhopman might know.
To answer your other question, I am not using deps query in bxl. I'm using all_actions. Is there a difference? TBH, I'm actually not sure why I have to use deps on the command line in the first place. My original intuition long time ago was that running buck2 aquery root//foo:bar would list all of the actions for that target. But it doesn't, it only lists one action of type run and category archive. So I just figured out through experimentation that if I use buck2 aquery deps(root//foo:bar) then I would get all of the actions. Example:
Yes there is a difference between deps and all_actions. all_actions is also available on the CLI. If you try all_actions query in the CLI, you should be getting the same thing as BXL. all_actions gives you all the actions of a given target, which is what I think you want? cc @cjhopman here as well since he implemented most of aquery.