Tup crashes msys tools on windows

[Qix-]

I know there has been a reported problem on Cygwin tools as well, but it has resurfaced with Msys tools now, too.

Running MinGW's gcc results in make.exe has stopped working.

Here is a dump of (some) of the output (sorry for the wall of text):

$ ../tup/tup.exe
[ tup ] [0.002s] Scanning filesystem...
[ tup ] [0.520s] Reading in new environment variables...
[ tup ] [0.520s] No Tupfiles to parse.
[ tup ] [0.523s] No files to delete.
[ tup ] [0.523s] Executing Commands...
 [    ]   0%tup error: Size of 1113340515 bytes is longer than the max filesize
 *** Command ID=256 failed: ar rcu "build\liblua.a" "build/obj\lapi.o build/obj\lauxlib.o build/obj\lbaselib.o build/obj\lbitlib.o build/obj\lcode.o b
uild/obj\lcorolib.o build/obj\lctype.o build/obj\ldblib.o build/obj\ldebug.o build/obj\ldo.o build/obj\ldump.o build/obj\lfunc.o build/obj\lgc.o build
/obj\linit.o build/obj\liolib.o build/obj\llex.o build/obj\lmathlib.o build/obj\lmem.o build/obj\loadlib.o build/obj\lobject.o build/obj\lopcodes.o bu
ild/obj\loslib.o build/obj\lparser.o build/obj\lstate.o build/obj\lstring.o build/obj\lstrlib.o build/obj\ltable.o build/obj\ltablib.o build/obj\ltm.o
 build/obj\lua.o build/obj\luac.o build/obj\lundump.o build/obj\lutf8lib.o build/obj\lvm.o build/obj\lzio.o" && ranlib build\liblua.a
 [    ]   0%tup error: Size of 258 bytes is longer than the max filesize
 *** Command ID=175 failed: gcc `gnustep-config --debug-flags` -fobjc-exceptions -std=gnu99 -Wall -Wextra -Werror -g2 -Isrc -Isrc/arch/i386 -Isrc/plat
/win32 -Ilib/lua -D_BLU_PLAT=win32 -D_BLU_ARCH=i386 -c "src/lua\blu_lua_state.m" -o "build/win32-i386/obj\blu_lua_state.o"
 [    ]   0%tup error: file_set_mtime() fstatat failed.
c:\BluDev\Blu\make.exe.stackdump: No such file or directory
* 1) gcc `gnustep-config --debug-flags` -fobjc-exceptions -std=gnu99 -Wall -Wextra -Werror -g2 -Isrc -Isrc/arch/i386 -Isrc/plat/win32 -Ilib/lua -D_BLU
_PLAT=win32 -D_BLU_ARCH=i386 -c "src/log-handler\blu_std_log_handler.m" -o "build/win32-i386/obj\blu_std_log_handler.o"
 *** tup errors ***
tup error: Unspecified output files - A command is writing to files that you didn't specify in the Tupfile. You should add them so tup knows what to e
xpect.
 -- Delete: make.exe.stackdump at dir 1
 *** Command ID=166 ran successfully, but tup failed to save the dependencies.
 [    ]  25%
 *** tup: 3 jobs failed.

Note that running the commands manually (literally copying them from the output above and running them) works just fine; it's definitely Tup's injection mechanism that's fouling things up.

Running Windows 7 64-bit.

As suggested in the previous ticket, I tried downgrading to version 7.2, with no luck. It doesn't appear to be an issue with the new 64-bit functionality, though I could be wrong. I would love to debug this, though I don't have MS Visual Studio installed.

---

Is there a way to add in a "just build it" flag that doesn't perform any filesystem checking when building on Windows? In my particular use case (and I'm sure I'm not the only one), I just need to build on windows for the sake of building on windows. I don't care if the source gets messed up because I'm not actually developing on windows.

It'd be nice to have at least some sort of compatibility mode that just works directly in the source tree, not playing around with the injection mechanism. So long as it's an optional flag to do so (since the injection mechanism works fine in a lot of cases), I don't think it'd interfere with the Tup philosophy.

Perhaps even a rule flag that specifies the command should be run in Windows compatibility mode:

: foreach *.c |> ^w^ gcc -c "%f" -o "%o" |> build/obj/%B.o

Where the ^w^ indicates that, on windows (and only on windows), the command should be (potentially unsafely) run without any hooking.

[gittup]

If you just want a single-time build, you can try to use the 'tup generate' command to get a shell script. The generated script will just run the commands without any dependency checking.

Any idea on the steps to reproduce this? I don't hit it with any of the toolchains I have on my win7-64 machine.

[FreddieChopin]

I'm having the same problem with my project so I may be able to provide a test case. This seems to be related to the length of path, because if I execute tup from "C:\ARM\distortosExamples" it works, but when I rename the folder to "C:\ARM\distortosExamplesLongerNameThanBefore", remove outputs and run tup again, it gives me this very cryptic message:

 [tup error: Size of 1096563267 bytes is longer than the max filesize
...
 *** Command ID=1240 failed: ./../../../../source/architecture/ARM/ARMv6-M-ARMv7
-M/ARMv6-M-ARMv7-M.ld.sh "STM32F407VG" "0x08000000,1048576" "0x20000000,131072"
"2048" "4096" "bkpsram,0x40024000,4096" "ccm,0x10000000,65536" > "../../../../..
/output\STM32F407VG.ld"

The output file that is created is correct.

I've tried that with the most recent version for Windows.

[gittup]

Thanks for the tip on the long paths! I was able to reproduce it with that info. There were actually two problems here - first, the event reader (windepfile.c) wasn't reading 0-length events correctly, since all events still have nul-terminators. This was throwing off the deserialization process, resulting in the wacky filesizes in the "Size of X bytes is longer than the max filesize" messages. This part should be fixed in commit 59fedf30a55ca3d32a6d5b5f02a18930cf1738cc.

The second problem was the fact that the dependency detection was generating 0-length events in the first place, which is where the long path issue comes in to play. This is because of the crap Windows APIs limiting things to 260 characters unless you use wide-character strings and the magic "?" prefix. I believe this part is fixed, or at least works slightly better, in commits 3c6fc31b7095dcd8b7c451c0ac0b212a64e850a2 and e54cf5961e76cdf526230ae3577a6ed5a83bbaa5. I only have one new test for this (t5082), which may only actually trigger the behavior in my test environment. So, if you find it's still not working correctly in your environment, please let me know as best you can how I can reproduce it and I can add more tests.