setup-dlang icon indicating copy to clipboard operation
setup-dlang copied to clipboard

Any idea how to integrate GDC?

Open p0nce opened this issue 4 years ago • 35 comments

Using a recent GDC would be a boon to our linux testing. Any idea how to do it?

p0nce avatar Dec 29 '20 15:12 p0nce

setup-dlang is more or less doing what the official install script is doing (only using Typescript + GH Actions builtins, instead of bash).

The main blocker is having prebuilt GDC packages, like LDC and DMD provide. IMO this should be done upstream either by the @D-Programming-GDC or @dlang organizations.

That said, technically nothing prevents someone for setting up a GitHub Action pipeline here that would do the same. Here's a starting point for a CI pipeline: https://github.com/D-Programming-GDC/gcc/blob/5ff6ab4f88164b4c8939a8780da42dc39bd2a7ff/.semaphore/semaphore.yml#L1

PetarKirov avatar Dec 30 '20 01:12 PetarKirov

Thanks. Maybe I'm being stupid: is there maybe a way to just invocate sudo apt install gdc in the user Action? Any GDC version would do.

p0nce avatar Dec 30 '20 10:12 p0nce

If you are fine with any version and only interested in Linux, literally running apt install gdc in your action should do the trick - and take roughly same amount of yml config :)

For setup-dlang there both needs to be some way to switch between version and support for all of Win-Linux-MacOS triplet. It should be possible to do, probably with some enhancements to GDC own CI, but it is a fair bit of work, much more than was needed for dmd/ldc, so it was not originally implemented.

mihails-strasuns avatar Dec 30 '20 11:12 mihails-strasuns

Fine! Thanks :)

p0nce avatar Dec 30 '20 11:12 p0nce

I think it would be better to keep this ticket open - both because it is a necessary feature request and because the same question is likely to come up again. I can create a new one if you prefer of course.

mihails-strasuns avatar Dec 30 '20 14:12 mihails-strasuns

We could just mention in the README how to use GDC.

Geod24 avatar Dec 31 '20 04:12 Geod24

Do you have any best practice suggestions? (other than apt install when applicable)

mihails-strasuns avatar Dec 31 '20 13:12 mihails-strasuns

apt install doesn't actually work in Gitub Actions for GDC.

  • On Ubuntu 20.04, the linker output an error that is: dub: symbol lookup error: dub: undefined symbol: _D3std3net4curl4HTTP9__mixin376onSendMFNdDFAvZmZv Similar to that thread: https://forum.dlang.org/post/[email protected]

  • Downgrading to Ubuntu 18.04, installing GDC removes dub and ldc packages.

image

But installing dub or ldc removes GDC: image

So I haven't found a way to support GDC properly... Any idea? Would need both DUB and GDC ideally.

p0nce avatar Jan 02 '21 23:01 p0nce

It may be a time to ask @ibuclaw for suggestions :)

mihails-strasuns avatar Jan 02 '21 23:01 mihails-strasuns

Mmmm, I follow the suggestions in the debian thread and install DUB from Github instead. This file allows to dub test with GDC:

name: gdc
on: [push, pull_request]

jobs:
    test:
        name: Dub Tests
        strategy:
            matrix:
                os: [ubuntu-20.04]
                dc: 
                    - ldc-latest 

        runs-on: ${{ matrix.os }}
        steps:
            - uses: actions/checkout@v2

            - name: Install D compiler
              uses: dlang-community/setup-dlang@v1
              with:
                  compiler: ${{ matrix.dc }}
          
            - name: Install DUB
              run: |
                  wget https://github.com/dlang/dub/releases/download/v1.23.0/dub-v1.23.0-linux-x86_64.tar.gz
                  tar xvf dub-v1.23.0-linux-x86_64.tar.gz

            - name: Install libcurl
              run: sudo apt install libcurl4-gnutls-dev

            - name: Install GDC
              run: sudo apt install gdc              

            - name: Vanilla unittest
              run: ./dub test -a x86_64 --compiler gdc -v

Not optimal but works for this single purpose.

p0nce avatar Jan 03 '21 00:01 p0nce

It may be a time to ask @ibuclaw for suggestions :)

I only see that the package maintainers have shot themselves in both feet.

  1. By making gdc and ldc conflict (did ldc install module sources to /usr/include/d on 18.04?)
  2. By insisting on using shared libraries by default when the phobos ABI is so unstable, building in a different path changes symbol names (yes, the debian/ubuntu builders use a generated /buildd path, so each minor version update probably requires all downstream packages to be rebuilt).

ibuclaw avatar Jan 03 '21 01:01 ibuclaw

I was able to use gdc on ubuntu-latest with this change: https://github.com/mbierlee/poodinis/pull/38/commits/6e25ca74fd7516efec48a6c3df92f99f5c296a42

essentially you need something like:

jobs:
  # On Ubuntu we can use GDC (so keep working for D version 2.076)
  gdc-latest:
    name: GDC on Ubuntu
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2

      - name: Install DMD (so dub is available)
        uses: dlang-community/setup-dlang@v1
        with:
          compiler: dmd-latest

      - name: Install GDC
        run: |
          sudo apt-get update
          sudo apt-get install gdc -y
          gdc --version

      - name: Build library
        env:
          DC: gdc
        run: dub build --compiler=gdc --build=release

SingingBush avatar Jun 09 '21 21:06 SingingBush

FWIW I can test both GDC 10 and GDC 12 in GH actions: https://github.com/AuburnSounds/intel-intrinsics/tree/master/.github/workflows

p0nce avatar Sep 07 '22 23:09 p0nce

Likewise: https://github.com/dlang/dub/pull/2324 Note that I added support for dub: version in this action, so you can simplify your configuration.

Geod24 avatar Sep 07 '22 23:09 Geod24

Can it be as simple as this?

      - name: Install D compiler
        uses: dlang-community/setup-dlang@v1
        with:
          compiler: gdc-latest

andrey-zherikov avatar Sep 08 '22 02:09 andrey-zherikov

This won't work currently. But yes, it would be good to make it work.

Geod24 avatar Sep 08 '22 06:09 Geod24

Unrelated? The godbolt DMD compiler are a bit old. I don't think you can test on the newest DMD.

p0nce avatar Sep 08 '22 13:09 p0nce

This won't work currently. But yes, it would be good to make it work.

As I see in compiler.ts, it's a matter of figuring out the URL for a specific GDC version, isn't it?. Is there an issue with this?

andrey-zherikov avatar Sep 08 '22 14:09 andrey-zherikov

updated example for multiple gdc versions:

## On Ubuntu we can use GDC. The compatibility of gdc is:
##   gcc gdc-10  -> D 2.076 (the default on Ubuntu 20.04 (ubuntu-latest), also available on 22.04)
##   gcc gdc-11  -> D 2.076 (requires Ubuntu 22.04)
##   gcc gdc-12  -> D 2.100 (requires Ubuntu 22.04)
## Note that Github's ubuntu-latest isn't actually the latest release. It's currently set to "ubuntu-20.04" despite github having "ubuntu-22.04" runners (similar situation with macos-latest)
## For this reason it's best to use ubuntu-22.04

jobs:
  test-gdc:
    name: ${{ matrix.compiler }} on ${{ matrix.os }}
    strategy:
      fail-fast: false
      matrix:
        os: [ ubuntu-22.04 ]
        compiler: [ gdc-10, gdc-11, gdc-12 ]
    runs-on: ${{ matrix.os }}
    steps:
      - uses: actions/checkout@v2

      - name: Install DMD (so dub is available)
        uses: dlang-community/setup-dlang@v1
        with:
          compiler: dmd-latest

      - name: Install dependencies on Ubuntu
        if: startsWith(matrix.os, 'ubuntu')
        run: sudo apt-get update && sudo apt-get install libev-dev libevent-dev libsqlite3-dev -y

      - name: Install ${{ matrix.compiler }}
        run: |
          sudo apt-get update
          sudo apt-get install ${{ matrix.compiler }} -y

      - name: Show version
        run: |
          ${{ matrix.compiler }} --version
          dub --version

      - name: Build library
        env:
          DC: ${{ matrix.compiler }}
        run: dub build --compiler=${{ matrix.compiler }} --build=release

SingingBush avatar Sep 09 '22 12:09 SingingBush

Just ran into this... Any help would be appreciated: https://github.com/dlang/undeaD/pull/56

schveiguy avatar Jun 09 '23 15:06 schveiguy

SingingBush's example is the current way how to do it. We don't have it in setup-dlang yet because it isn't supported on platforms other than ubuntu-2022.

What we could do is just running apt install on ubuntu and only allow that as the only supported GDC OS on setup-dlang for now though.

WebFreak001 avatar Jun 09 '23 19:06 WebFreak001

OK, so given that I already have a matrix of oses and compilers, how do I skip the invalid combinations? Or do I just make a new yml file?

schveiguy avatar Jun 09 '23 20:06 schveiguy

you can use exclude to exclude all the things that match (entire combinations or also just partial matches iirc)

See https://docs.github.com/en/actions/using-jobs/using-a-matrix-for-your-jobs#excluding-matrix-configurations

WebFreak001 avatar Jun 09 '23 20:06 WebFreak001

What's about adding configuration?

      matrix:
        os: [ubuntu-latest, windows-latest, macOS-latest]
        dc: [dmd-latest, ldc-latest]
        include:
          - os: ubuntu-latest
            dc: gdc-latest

andrey-zherikov avatar Jun 09 '23 20:06 andrey-zherikov

you need to separate the installation step like for example D-Scanner does it then: https://github.com/dlang-community/D-Scanner/blob/5a53c538d0aa832f03840840271b6631fbbfc53d/.github/workflows/default.yml#L78-L89

WebFreak001 avatar Jun 09 '23 20:06 WebFreak001

where does DC get defined? I got the workflow running, it installs gdc (and dub from dmd), and then runs dub with dmd, because apparently DC is set to dmd.

schveiguy avatar Jun 09 '23 20:06 schveiguy

oh yeah setup-dlang sets DC - you can manually set it (it's just an environment variable) - or use it to only install dub.

WebFreak001 avatar Jun 09 '23 21:06 WebFreak001

will setup-dlang set DC if I set it first? (hoping it doesn't)

schveiguy avatar Jun 09 '23 21:06 schveiguy

yes, it does so here: https://github.com/dlang-community/setup-dlang/blob/43589c229861e1720e187a344c67dad1d9eefe4c/src/main.ts#L64

WebFreak001 avatar Jun 09 '23 21:06 WebFreak001

you need to separate the installation step like for example D-Scanner does it then: https://github.com/dlang-community/D-Scanner/blob/5a53c538d0aa832f03840840271b6631fbbfc53d/.github/workflows/default.yml#L78-L89

Can setup-dlang take care of this? Even if it's restricted to ubuntu only, this would be very helpful.

andrey-zherikov avatar Jun 09 '23 21:06 andrey-zherikov