Full Windows Support

[John-Colvin]

clang say they mostly match the msvc C++ ABI on windows (https://clang.llvm.org/docs/MSVCCompatibility.html) so it should be possible to get everything working nicely just like on linux.

• [x] get dub tests to pass (dub test)
• [ ] get cucumber tests to pass (bundle exec cucumber --tags ~@wip)
• [x] get appveyor working (can probably copy https://github.com/jacob-carlborg/dstep for the annoying stuff e.g. getting llvm)
• [ ] get #include <nn.h> (nanomsg) to work and see if you can replace the bindings at https://github.com/symmetryinvestments/nanomsg-wrapper and have the tests there still pass on windows

[atilaneves]

@adamdruppe - I tried assigning this to you but couldn't.

[adamdruppe]

maybe it is cuz i never forked the thing, try it now

[adamdruppe]

quick update, I have it down to just 5 tests still failing. Kinda surprising that one of these would be Windows-specific, just a ctor decl outside the class (issue 115). But still getting through it all.

[adamdruppe]

So... clang --ast-dump agrees with what I'm seeing in dpp. The compound statement is included on linux, but just null on Windows. I tried a new llvm too, no change. It could be a different build flag or something, but if people are going to use the upstream Windows binaries, I think we just need to work around this.

The good news is a member function definition is.... maybe something you never actually need? Unless it is trying to automatically port a template body from c++ over to D instead of just matching mangling which I don't see even Linux version attempting.. but idk, since the clang command directly is showing the same hole as found in the code it makes me fairly certain it is a problem in the upstream library/build.

[adamdruppe]

see https://github.com/atilaneves/dpp/pull/183 for checking off the dub test. I haven't tried the others yet, my Windows box doesn't have any ruby so that will be more painful to run there.

[adamdruppe]

i just got ruby installed on my windows box. 10 scenarios, 7 failed, 3 passed. 66 steps, 7 failed, 22 skipped, 37 passed.

at least it went quickly after getting all the deps installed! i'll start looking at them now.

edit: lol they assume g++ is in path. fun.

[adamdruppe]

OK, good news on that. The tests all fail because of trivial name changes like .obj instead of .o and files being in PATH. I wrote a helper program to translate those.

import std.process;
import std.algorithm;
import std.array;

string fixObjFiles(string a) {
	version(Windows) {
		if(a.endsWith(".o")) return a[0 .. $-2] ~ ".obj";
		if(a.startsWith("-of")) return a ~ ".exe";
	}
	return a;
}

string fixCompiler(string a) {
	version(Windows) {
		if(a == "gcc" || a == "g++")
			return "clang"; // this box has clang specifically for dpp sooooo
		if(a.startsWith("./")) {
			import std.file;
			import std.path;
			return buildNormalizedPath(thisExePath, "../../tmp/aruba", a ~ ".exe");
		}
	}
	
	return a;
}

int main(string[] args) {
	if(args[1] == "dmd" || args[1] == "d++")
		args ~= "-m64";
	auto sargs = args[1 .. 2].map!fixCompiler.array ~ args[2 .. $].map!fixObjFiles.array;
	import std.stdio; writeln("RUNNING ", sargs);
	auto p = spawnProcess(sargs);
	return wait(p);
}

hacky but like it is a good sign these only failed due to trivialities too.

the only other one is the

Scenario: Compile but don't link When I successfully run fixup d++ -c main.dpp Then a file matching %r<main.[o|obj]> should exist

and that regex fixes it.

prolly time to actually try using it... I suspect it will work now.

[John-Colvin]

Excellent, this is looking very promising.

[adamdruppe]

i just ran nanomsg on Linux literally seconds ago using dpp instead of the bundled bindings (though i think the bundled bindings are a better approach...). Took a few semi-hacky things, undef VERSION and errno, and I ran dpp separately from dub (but I suppose preBuildCommands could run it automatically there i don't feel like that is a good solution)... but it worked.

I'm gonna copy the files to my Windows box soon and see if it works there as well.

[adamdruppe]

the include path on Windows isn't automagically figured out... but i can add it manually (took me forever to even find it but meh). and now the .lib has leading _ and D doesn't put that out.

again, it is easy enough to convert the lib but just another thing that doesn't just work... i wonder if there's any good way to know why it has it or if it does etc

[adamdruppe]

OK, nanomsg did work on Windows, I just had a 32 bit build and making dpp do that i guess --clang-option -m32 works but idk if it is the best.

Replaced the import nanomsg.bindings with

#include <nanomsg/nn.h>
#include <nanomsg/reqrep.h>
#include <nanomsg/pubsub.h>
#include <nanomsg/pipeline.h>
#include <nanomsg/pair.h>
#include <nanomsg/survey.h>
#include <nanomsg/bus.h>
#include <nanomsg/tcp.h>
#undef errno
#undef __VERSION__

but the rest was just setup pain. The undefs are interesting as they conflicted with D stuff... but solvable right there. so I'd say dpp does work on Windows you just need to tell it where to find third party lib files.

[atilaneves]

The appveyor build works now.

The more I think of it the less I seem to care about getting cucumber to work on Windows. It's probably better to rewrite the bash scripts that test external projects into D actually.

[adamdruppe]

What are you next steps? I think it would be kinda nice if dub actually understood .dpp files in the source and just automatically read in dpp as a dependency and pulled it down.

[John-Colvin]

I worked on that a while ago. I will try to resurrect next week, or at least point you to what I started

Regards, John Colvin

On Fri, 18 Oct 2019, 23:10 Adam D. Ruppe, [email protected] wrote:

What are you next steps? I think it would be kinda nice if dub actually understood .dpp files in the source and just automatically read in dpp as a dependency and pulled it down.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/atilaneves/dpp/issues/182?email_source=notifications&email_token=AAOH3NM3BMSV5SNPQEGFHBDQPIX5TA5CNFSM4I7AC7XKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEBWGMCQ#issuecomment-543974922, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAOH3NLXZJJJZFBMDBZMMXTQPIX5TANCNFSM4I7AC7XA .

[atilaneves]

What are you next steps?

I don't have any as far as Windows support goes. appveyor will keep us honest.

[Imperatorn]

Schrödinger's Windows support? Is it working now, or not? Is it just the cucumber support that's blocking it?

[aminya]

I have been trying to build a library but it fails with strange errors such as the following on Windows. The same code works on Ubuntu!

> dub run dpp -- ./file.dpp ../../dist/build/CMakeFiles/file.dir/dist/src/file.c.obj

..\..\dist\build\CMakeFiles\file.dir\dist\src\file.c.obj Offset 00000H Record Type 0064
 Error 138: Module or Dictionary corrupt
Error: linker exited with status 1

I also tried the solution in this comment, the error message is the same.

❯ ./fixup.exe C:/Users/aminy/AppData/Local/dub/packages/dpp-0.4.4/dpp/bin/d++.exe ./file.dpp ../../dist/build/CMakeFiles/file.dir/dist/src/file.c.o
RUNNING ["C:/Users/aminy/AppData/Local/dub/packages/dpp-0.4.4/dpp/bin/d++.exe", "./file.dpp", "../../dist/build/CMakeFiles/file.dir/dist/src/file.c.obj"]
Error: Could not execute `dmd ../../dist/build/CMakeFiles/file.dir/dist/src/file.c.obj .\file.d -of./file.exe`:
OPTLINK (R) for Win32  Release 8.00.17
Copyright (C) Digital Mars 1989-2013  All rights reserved.
http://www.digitalmars.com/ctg/optlink.html
..\..\dist\build\CMakeFiles\file.dir\dist\src\file.c.obj Offset 00000H Record Type 0064
 Error 138: Module or Dictionary corrupt
Error: linker exited with status 1

Same thing when I use the obj extension instead of o when running the command.

❯ ./fixup.exe C:/Users/aminy/AppData/Local/dub/packages/dpp-0.4.4/dpp/bin/d++.exe ./file.dpp ../../dist/build/CMakeFiles/file.dir/dist/src/file.c.obj
RUNNING ["C:/Users/aminy/AppData/Local/dub/packages/dpp-0.4.4/dpp/bin/d++.exe", "./file.dpp", "../../dist/build/CMakeFiles/file.dir/dist/src/file.c.obj"]
Error: Could not execute `dmd ../../dist/build/CMakeFiles/file.dir/dist/src/file.c.obj .\file.d -of./file.exe`:
OPTLINK (R) for Win32  Release 8.00.17
Copyright (C) Digital Mars 1989-2013  All rights reserved.
http://www.digitalmars.com/ctg/optlink.html
..\..\dist\build\CMakeFiles\file.dir\dist\src\file.c.obj Offset 00000H Record Type 0064
 Error 138: Module or Dictionary corrupt
Error: linker exited with status 1

[aminya]

I found a workaround. Only use dpp to generate the required d files, then use dmd for linking.

This first generates the required d files:

dub run dpp -- --keep-d-files ./file.dpp --preprocess-only

Then compile the generated d file using dmd passing -m64

dmd ../../dist/build/CMakeFiles/file.dir/dist/src/file.c.obj ./file.d -of=./file.exe -m64

The issue seems to be in the linker that dpp use underneath.