DPP Windows support for MongoC and Mondo

[Laeeth]

Related to #182 Two ways to use Mongo DB in D - vibe and mondo. Problem is that pulling in vibe means pulling in quite a lot and tends to be fragile and break with new releases of DMD. Furthermore, vibe static constructor interacts badly with D bug in running static constructors twice for shared library on Linux and some entirely different bug on Windows (ask John).

For the time being, it's necessary for us to talk to Mongo even though it's by far not my cup of tea personally.

So I'd like to use Mondo to do this.

However Mondo is built using a hacked-together script around dstep and this only works on Linux.

I forked Mondo and replaced the dstep build script with something much simpler based on dpp. That works for Linux and ought to work on Windows, but dpp didn't quite work and now does thanks to @adamdruppe @veelo and others.

Since it's an important use case, if you are agreeable then would like to have this be one of the integration tests for dpp. If feel strongly against, then fine - the carbon integration tests should catch problems.

Sequence:

1. Two below depends on mongoc and bsonc libraries. See the docs for those on how to build on Windows - it uses cmake so should not be that bad. Make sure at least you can build these on Windows and make a script to clone the original repos (we should be able to choose the branch/tag we want), build them and install in relevant subdir of extern_deps

2. https://github.com/symmetryinvestments/mondo Make sure this works on Windows. Ignore the dstep based script and uncomment the preGenerate lines in the dub.sdl.

3. Make sure we have a new release of dpp that actually incorporates Windows support. Make sure README is up to date - that it works on Windows and how to use it.

4. Announce on forums saying work has been sponsored by Symmetry.

5. Send PR to upstream mondo explaining decisions and that it adds Windows support. PR should also update README so it's clear.

@adamdruppe @atilaneves @John-Colvin

[adamdruppe]

dpp takes ages to build those files. I can't really recommend having users go through that; really I'd bundle the C lib and the translation ahead and only run the script if those versions change.

I don't know why it is so slow, my guess is just going through a LOT of include files - there's an #include<windows.h> in there.... heck an #include<algorithm> too but windows.h is huge.

[adamdruppe]

it might be a good idea to make dpp actually recognize windows.h and skip it - just replace that with the premade core.sys.windows.windows in D, unless an override switch is given.

[adamdruppe]

yeah an isolated test confirms windows.h itself is the brutal slowness. gotta do something about that. i just don't know what yet.

[adamdruppe]

alias PMETARECORD = __unaligned tagMETARECORD*;

it generated that trying to do a windows.h test too, which is a syntax error.

and this too

    static if(!is(typeof(PROFILE_EMBEDDED))) {
        enum PROFILE_EMBEDDED = 'MBED';
    }

prolly msvc extensions. but it also took like an hour to generate this file,

[Laeeth]

Fine near term if we vendor Windows. Being able to do this at all allows making Carbon a plug-in which opens up a lot.

Longer term yes I agree. @aneves ?

[Laeeth]

I guess DPP needs simple instrumentation too ?

[adamdruppe]

so while lying in bed last night i thought of a few potential solutions. first i want to change the thing to enable those extensions. so i'll fix that in dpp so windows.h does work.

but then I plan on making it just skip windows.h. clang can parse it quickly enough, so it will do that, but no need to output 30,000 lines of translated D jsut because it was referenced. just importing core.sys.windows.windows ought to work in 99% of cases.

i work slowly on windows but i should have this finished later today.

[atilaneves]

so while lying in bed last night i thought of a few potential solutions. first i want to change the thing to enable those extensions. so i'll fix that in dpp so windows.h does work.

but then I plan on making it just skip windows.h. clang can parse it quickly enough, so it will do that, but no need to output 30,000 lines of translated D jsut because it was referenced. just importing core.sys.windows.windows ought to work in 99% of cases.

i work slowly on windows but i should have this finished later today.

Easiest is to ignore windows.h via the command-line, similarly to the existing options to ignore namespaces.

[adamdruppe]

On Thu, Oct 24, 2019 at 06:29:02AM -0700, Atila Neves wrote:

Easiest is to ignore windows.h via the command-line, similarly to the existing options to ignore namespaces.

is this already there?

[atilaneves]

On Thu, Oct 24, 2019 at 06:29:02AM -0700, Atila Neves wrote: Easiest is to ignore windows.h via the command-line, similarly to the existing options to ignore namespaces. is this already there?

No, but is very easy to add.

[adamdruppe]

On Thu, Oct 24, 2019 at 08:20:55AM -0700, Atila Neves wrote:

No, but is very easy to add.

wanna give a hint, I don't see an immediate answer.

two options i see are checking SourceLocation but that sucks and then the clang has a clang_getIncludedFile function so presumably I can check that but don't see the best place to put it yet

[aneves]

Longer term yes I agree. @aneves ?

@Laeeth, not the user you meant. :) cheers

[adamdruppe]

eh checking on path kinda works but gotta be careful not to exclude stuff that is used while skipping stuff that are in different files but down line.

the cursors are already preprocessed by the time it gets in there :(

i'm kinda tempted to only translate symbols actually referenced in the d file. i wonder if that would work for any real world use.