--cwd in --each/--next causes errors if there is a --cwd outside of --each

[theJenix]

tl;dr If you use a haxe hxml file that contains a --each/--next, and inside those build steps you include -C to change the working folder for that step, it will fail with a Invalid Directory error if haxe is invoked with a -C outside the build.hxml. I think this is because it attempts to apply the outer -C to each build step, wherein it will only succeed before the first one.

Details:

This may be an issue that is specific to my setup, but it feels like a bug or gap all the same.

I am using Vscode/vshaxe in a large mono repo with many different apps and libraries My haxe.configurations setting has several entries that look something like ["-C", "lib/platform", "build.hxml"], ["-C", "app/widget", "build.hxml"],

which enables me to switch the active configuration in Vscode and build/develop/debug each library or app separately. I was trying to reorganize my apps folder so they would contain a server component and a client component, e.g.

app/widget/server
app/widget/client

each of which has a build.hxml

and so my hxml file looks like:

--each
--cwd server
build.hxml

--next 
--cwd ../client 
build.hxml

When I run haxe build.hxml from the project folder, everything works fine.

If go up a directory and then run haxe -C widget build.hxml, the first build step runs and then it fails with a Error: Invalid Directory: widget. error. This therefore means that my strategy for maintaining haxe.configurations does not work in this case, even though it feels like it should.

[Simn]

Reading this makes my head hurt... I don't even understand how your example is supposed to work from the project directory because Haxe resets the cwd to the original directory when coming across --next.

I think we'll need a standalone project demonstrating the issue.

[theJenix]

Reading this makes my head hurt...

Sorry! My wife says I have that affect on people... :)

I don't even understand how your example is supposed to work from the project directory because Haxe resets the cwd to the original directory when coming across --next.

Actually it doesn't, and I think that's the issue. I would expect Haxe to reset cwd to the directory it was in before the --each was encountered before entering --next, but the cwd seems to leak thru the --each/--next blocks

I think we'll need a standalone project demonstrating the issue.

archive.zip

Unzip and change to the proj subfolder: haxe build_error.hxml produces an error haxe build_working.hxml completes without error the difference between them is the --cwd inside the --next block

I also included build files up one level that better mimics what I was trying to do originally; these are copies of the hxml files in proj that have a --cwd proj at the top of the file (above the --each)

haxe build_error.hxml produces the same error as above haxe build_different_error.hxml produces a similar but different error Error: : Invalid directory: proj which seems related to this whole thing.

This was all observed on a 2021 MacBook Pro 14" M1 Max, using Haxe 4.3.2 installed thru home-brew.

Does that help?

[Simn]

Thanks, that makes it clearer! So the trick here is that there's a --cwd before the --each. I missed that in the original description, and I agree that in such cases the --next should reset to that one.

[theJenix]

Awesome; sorry for the initial confusion! That is definitely the issue I tried to report originally, but I now think it's actually a bit bigger than that. The first part of my standalone project description doesn't include a --cwd before the --each, and the error implies that the cwd isn't being reset between --next blocks even without a --cwd set before the --each.

It looks like your fix definitely addresses the ---cwd before --each case (including fixing that weird error I got, i.e. the build_different_error case described above), but does it also handle this case where we don't supply a --cwd before --each but still expect cwd to be reset after --each and --next?

Thanks!

[Simn]

Yes, but good call, I've added a test for this case as well to make sure.

[Simn]

Reopening because the tests for this issue expose a problem related to the compilation server. I think what happens is that the server is unaware of the cwd change, and then resolves a module path ([], "Main") to the same module no matter where we're starting from.

There are some checks already for when the directories themselves change, but the current working directory seems untracked. I'm not sure how it should react either because in theory a cwd change could affect the entire cache, but essentially disabling caching on --cwd wouldn't be very practical.

[theJenix]

Is it reasonable that the cache would just be bucketed by cwd? It would be safe, at the potential cost of inefficiencies on startup (since you'd have to fill n caches for n --cwd statements). It would also be inefficient a compilation could truly be shared across cwd's and memory cost would scale with cwd calls. But definitely better than just disabling cache.