SDL icon indicating copy to clipboard operation
SDL copied to clipboard

Published 20 hours ago • verified publisher icon libsdl-org

Dump buildbot for GitHub Actions.

Open icculus opened this issue 3 years ago • 25 comments

Task list

This is what the buildbot covered, plus some other wishlist items. Some we may want to drop (QNX? Windows Phone?), others might be covered in other ways (for example, the multiple macOS architectures might all be caught by the Xcode work, but we might still want to make sure we have configure and cmake coverage on those platforms generally, so these don't all have to be separate GitHub Actions).

On every commit/pull request

  • [x] ~~Linux/amd64, configure~~ (no longer in SDL3)
  • [x] Windows/amd64, CMake
  • [x] macOS/amd64, CMake

Occasionally, on a schedule

  • [x] Android/arm64
  • [x] Emscripten/wasm
  • [x] FreeBSD/amd64
  • [x] Haiku/amd64
  • [x] iOS
  • [ ] Linux/x86
  • [ ] Linux static analysis
  • [x] macOS/amd64
  • [x] macOS/arm64
  • [ ] macOS with xcode project files
  • [ ] macOS static analysis
  • [x] mingw64/amd64
  • [x] mingw64/x86
  • [ ] OpenBSD/amd64
  • [x] OS/2 with OpenWatcom
  • [x] ~~QNX/amd64~~ (still available in SDL3 but a builder is impractical, see comments.)
  • [x] ~~QNX/x86~~ (still available in SDL3 but a builder is impractical, see comments.)
  • [x] ~~QNX/arm64~~ (still available in SDL3 but a builder is impractical, see comments.)
  • [x] ~~QNX/arm32~~ (still available in SDL3 but a builder is impractical, see comments.)
  • [ ] Raspberry Pi
  • [x] WinRT/UWP
  • [x] Windows/amd64 with Visual Studio project files
  • [x] Windows/x86 with Visual Studio project files
  • [x] ~~Windows Phone/arm32~~ (no longer offered in SDL3)
  • [x] PSP
  • [x] PS Vita
  • [x] Nintendo 3DS
  • [x] RiscOS
  • [x] CMake where possible
  • [x] ~~configure script where possible~~ (no longer offered in SDL3)
  • [ ] static analysis where it dramatically improves code coverage (Windows, Linux, macOS...others less so.)
  • [ ] upload builds where possible

Original comments

(This is just a brain dump for now.)

This is not for the 2.0.18 milestone, but I've been thinking about the time I spend making sure the buildbots continue to stay up and running, and watching things like the PS Vita port get added to GitHub Action's CI system with a few lines of configuration, and wondering why we're messing with this thing and not just making it another service we moved to GitHub so it Just Works and is Nicely Integrated.

Also, even though the buildbot master is hosted on libsdl.org, the actual builders are on a workstation in my office; if I get hit by a bus tomorrow, the builders last until the next power outage, or Windows update automatic reboot, or whatever. Since this is some beefy hardware, running about a bazillion vmware instances, it wasn't really technically or financially feasible to run it on Digital Ocean with the rest of libsdl.org.

Some of the Buildbot builders are obviously redundant now--Linux, macOS, Windows, even Emscripten--and can be shut off right now in favor of the GitHub Actions counterparts...but there are some useful tools and esoteric platforms on Buildbot that are less trivial to duplicate on GitHub.

Notably:

  • Static analysis isn't hooked up to a GitHub Action, and it's worth noting that on pretty beefy hardware, this takes quite some time per-commit...it's not uncommon for static analysis to take 8-10 minutes, but it's also not clear how much of that is competing for CPU time verses other builders and virtual machines, and would be less of an issue on GitHub's server farm. Once you cook through all of GitHub's free minutes, you have to pay for CPU time, and this might be the sort of thing that pushes it over the limit, though. In practice, we could limit static analysis to just pushes to main and not pull requests (although it would be useful especially on pull requests!). We might have to cross this bridge once we come to it. Right now we have static analysis on macOS and Linux, but I would love to get it on as many platforms as humanly possible. The more code coverage you can get, the more free proactive debugging labor this thing generates. In practice, this should work anywhere that has a reasonable Clang install.
  • Less mainstream platforms: right now we have FreeBSD, OpenBSD, and Haiku in vmware on the buildbot, but these platforms are not available on GitHub's servers. GitHub has a tool where you run your own builders but otherwise hook into GitHub Actions, but we'd have to turn off Pull Requests for those platforms (otherwise these can run arbitrary code, which we don't care about on GitHub because the OS instance has no meaningful data to steal or live on a private LAN and is blown away after each build), and crucially, we'd have to continue to host the builders. Also, GitHub's tool doesn't work on any of these platforms, although apparently this third-party app is an option: https://github.com/ChristopherHX/github-act-runner ...but it might be better to move these to cross-compilers on a standard Linux install or something and hope it's "close enough" with a Haiku sysroot and not a full Haiku install.
  • Some of these are building weird targets on a standard Linux install; the OS/2 port is built with the Linux version of OpenWatcom, for example. This part is absolutely doable, but we'd have to figure out how to get those tools in-place on each iteration. That's a matter of research but probably isn't hard.
  • QNX's builds can be done on (Intel) macOS or Linux, but it's with non-public tools and requires a license file, so that's absolutely hitting the end of the line here, as we probably can't get that onto GitHub's servers in a reasonable way. But as I've said recently, it's probably time to ditch the QNX builds, as the tools I have are years out of date now.
  • Several (but not all) targets on the buildbot upload binaries for almost every commit, and I think it's useful to be able to tell people "these are presented as-is, but if you just need a DLL fast, you can grab it here," and I'd like to keep that if we dump buildbot (but doing so is not a dealbreaker for me). Likewise for uploading the static analysis HTML output; if that can't be hosted at GitHub, we'll need to upload it to libsdl.org somehow for viewing.

Anyhow, these are things to think about. In a perfect world, we have all the existing functionality migrated to GitHub Actions and the buildbot turned off in the 2.0.20 timeframe, but we'll have to see what our options are.

icculus avatar Sep 17 '21 23:09 icculus

Some of the Buildbot builders are obviously redundant now--Linux, macOS, Windows, even Emscripten--and can be shut off right now in favor of the GitHub Actions counterparts...

Not exactly true: Buildbot uses autotools, Github Actions only use CFake: Github actions must be updated to also use autotools (which would most importantly cover MinGW), then you can consider buildbot redundant.

sezero avatar Sep 19 '21 01:09 sezero

Using the configure scripts wouldn't be a problem, I think; I assume we used CMake on GitHub Actions because it was what the original implementor chose, but also because one small config could handle a bunch of platforms. But I assume changing this is easy enough.

(If we want to get wild, we could have it build the appropriate platforms with both to catch build project bugs.)

icculus avatar Sep 19 '21 03:09 icculus

(If we want to get wild, we could have it build the appropriate platforms with both to catch build project bugs.)

That would be preferable

sezero avatar Sep 19 '21 03:09 sezero

Several (but not all) targets on the buildbot upload binaries for almost every commit, and I think it's useful to be able to tell people "these are presented as-is, but if you just need a DLL fast, you can grab it here," and I'd like to keep that if we dump buildbot

Github Actions can upload artifacts to Github as part of the build process. Here's an example of that in action, at the bottom of the page here: https://github.com/love2d/love/actions/runs/1203406489 (I imagine it wouldn't be hard to upload HTML files when applicable, too).

However I'm not sure if there's an easy way to grab the latest artifact URLs from a certain branch, or to take a commit hash as input and get the github actions and artifact URLs from that. Maybe there are other tools which can do that...

slime73 avatar Sep 19 '21 03:09 slime73

for HTML files you could push to a new repository and make GH Pages activated there, so you can read directly.

ericoporto avatar Sep 19 '21 12:09 ericoporto

However I'm not sure if there's an easy way to grab the latest artifact URLs from a certain branch, or to take a commit hash as input and get the github actions and artifact URLs from that. Maybe there are other tools which can do that...

There is also the issue that GitHub Actions artifacts are only available to people who have logged in to GitHub.

The solution to both of these issues is to publish a new release from Actions for each run, which is also possible to do.

InfoTeddy avatar Sep 28 '21 06:09 InfoTeddy

Some of these are building weird targets on a standard Linux install; the OS/2 port is built with the Linux version of OpenWatcom, for example. This part is absolutely doable, but we'd have to figure out how to get those tools in-place on each iteration. That's a matter of research but probably isn't hard.

I took care of this one. The magic is the "actions/cache@v2" thing which basically makes the OpenWatcom install instantaneous if we've successfully run before. If the cache is removed--which happens periodically--we fall back to installing it again, which is also not slow since it's just unzipping a file downloaded from GitHub's web servers.

We can tweak build-scripts/os2-buildbot.sh to use binl64 instead of binl (the buildbot was running on the 32-bit Linux install, just to spread builds across virtual machines more evenly, but this isn't an issue with GitHub Actions), but otherwise, OS/2 is hooked up and good to go now.

Anyhow, I just wanted to try this out real fast; I'll hit GitHub Actions hard after 2.0.20 is finalized in the next few days.

icculus avatar Jan 04 '22 03:01 icculus

We can tweak build-scripts/os2-buildbot.sh to use binl64 instead of binl

Done in ae9e2149a55c75e99a2d1dade0f2c3a031b9139d.

icculus avatar Jan 12 '22 02:01 icculus

I think we're going to abandon QNX. It needs a commercial license key and I don't think that's something we can get away with in a public repo, since we'd have to put it and the SDK someplace visible for the build machine to download it.

Also my SDK is years out of date.

Obviously we'll still accept patches and aren't removing anything, we just won't have a builder for it.

icculus avatar Jan 15 '22 04:01 icculus

@pionere has a patch for build artifacts; apply this or something like this before we're done: https://github.com/pionere/SDL/commit/bf08b8f89ea2e37dee0d7e63386116e763f5eff6

icculus avatar Jan 24 '22 14:01 icculus

So something that Sam and I discussed: we really want the per-push and per-pull-request actions to just be a fast check...maybe just Linux/Mac/Windows in a reasonable default, so we can see right away if a commit is obviously incorrect in the common cases.

GitHub Actions lets you run things on a schedule (literally using cron * * * * * syntax!), so it would be useful to use that once a day and just go nuts with tons of time-consuming builds...all the platforms we can, static analysis, nightly build artifacts, etc. So once a day, or once a week, whatever, if there was a commit to check, it'll cook through everything and we can clean up the trash in the morning.

icculus avatar Feb 01 '22 17:02 icculus

@icculus : you might want to look at this example. It checks every day if there was a commit in 24 hours, and runs only if necessary (should_run).

pionere avatar Feb 01 '22 18:02 pionere

Focusing on adding all these targets is going to take a lot of time, and I'm not unhappy with the current GitHub Actions state for now, so I'm going to bump this from 2.24.0.

icculus avatar Jul 22 '22 02:07 icculus

Hi, I think the above list can be updated as more boxes are ticked.

One thing that I am curious, the assets that are on the release, are they built in the buildbot? I was trying to reproduce the SDL2-devel-2.24.1-VC.zip and I went looking for what config is used and I could not find this in the actions.

The reason of the asking is we are trying to figure out why a locally built SDL2 (using VS solution) works under Wine on MacOS, but the one available on GitHub release doesn't. Opened it's own issue #6996 .

ericoporto avatar Dec 10 '22 21:12 ericoporto

I'm glad I put this into a PR to check because it actually caught a build error, but whoa, that 49 minute build time on the Haiku@QEMU runner is brutal. We have to do something else with that.

Originally posted by @icculus in https://github.com/libsdl-org/SDL/issues/7673#issuecomment-1533736480

(It went up to 115 minutes for a successful build.)

Can I just set up a little Intel machine in my office and make it a dedicated Haiku builder, and have just the Haiku builds directed there instead of using GitHub's build farm? Or can we set up a cross-compiler that runs natively on Linux? The builder isn't useful if it takes 2 hours to build a project that can be compiled in under a minute normally.

icculus avatar May 04 '23 02:05 icculus

One thing that I am curious, the assets that are on the release, are they built in the buildbot? I was trying to reproduce the SDL2-devel-2.24.1-VC.zip and I went looking for what config is used and I could not find this in the actions.

(Sorry, I missed this comment before.)

They aren't built on the buildbot (or GitHub Actions); @slouken builds them for official releases on a local machine, afaik.

icculus avatar May 04 '23 02:05 icculus

FreeBSD (also a virtual machine here) took 15 minutes, which is better than 2 hours, but still. :)

icculus avatar May 04 '23 02:05 icculus

Let's make the haiku workflow workflow_dispatch-only then. I think there is still value in keeping the workflow available to verify whether it still builds, but let's comment out the pull_request trigger.

madebr avatar May 04 '23 03:05 madebr

Let's make the haiku workflow workflow_dispatch-only then.

Agreed. But being one of the more unique targets, there really is a value to it building by default, if we can dramatically lower that build time at some point in the future.

icculus avatar May 04 '23 03:05 icculus

It's disabled in fe68fd3af. I just tried a haiku cross compiler in this docker container (haiku/cross-compiler:x86_64), but it fails to build with the following errors:

[2/189] Building CXX object CMakeFiles/SDL3-shared.dir/src/misc/haiku/SDL_sysurl.cc.o
FAILED: CMakeFiles/SDL3-shared.dir/src/misc/haiku/SDL_sysurl.cc.o 
/bin/x86_64-unknown-haiku-g++   -DSDL3_shared_EXPORTS -DSDL_BUILD_MAJOR_VERSION=3 -DSDL_BUILD_MICRO_VERSION=0 -DSDL_BUILD_MINOR_VERSION=0 -DUSING_GENERATED_CONFIG_H -Iinclude-config- -I/src/src -Iinclude -I/src/include -I/src/include/SDL3 -std=c++11 -fPIC -fvisibility=hidden   -D_REENTRANT -Wall -Wno-multichar -Wundef -fno-strict-aliasing -Wshadow -fdiagnostics-color=always -MD -MT CMakeFiles/SDL3-shared.dir/src/misc/haiku/SDL_sysurl.cc.o -MF CMakeFiles/SDL3-shared.dir/src/misc/haiku/SDL_sysurl.cc.o.d -o CMakeFiles/SDL3-shared.dir/src/misc/haiku/SDL_sysurl.cc.o -c /src/src/misc/haiku/SDL_sysurl.cc
/src/src/misc/haiku/SDL_sysurl.cc: In function 'int SDL_SYS_OpenURL(const char*)':
/src/src/misc/haiku/SDL_sysurl.cc:30:30: error: 'class BUrl' has no member named 'OpenWithPreferredApplication'
     const status_t rc = burl.OpenWithPreferredApplication(false);
                              ^
[8/189] Building CXX object CMakeFiles/SDL3-shared.dir/src/video/haiku/SDL_bopengl.cc.o
FAILED: CMakeFiles/SDL3-shared.dir/src/video/haiku/SDL_bopengl.cc.o 
/bin/x86_64-unknown-haiku-g++   -DSDL3_shared_EXPORTS -DSDL_BUILD_MAJOR_VERSION=3 -DSDL_BUILD_MICRO_VERSION=0 -DSDL_BUILD_MINOR_VERSION=0 -DUSING_GENERATED_CONFIG_H -Iinclude-config- -I/src/src -Iinclude -I/src/include -I/src/include/SDL3 -std=c++11 -fPIC -fvisibility=hidden   -D_REENTRANT -Wall -Wno-multichar -Wundef -fno-strict-aliasing -Wshadow -fdiagnostics-color=always -MD -MT CMakeFiles/SDL3-shared.dir/src/video/haiku/SDL_bopengl.cc.o -MF CMakeFiles/SDL3-shared.dir/src/video/haiku/SDL_bopengl.cc.o.d -o CMakeFiles/SDL3-shared.dir/src/video/haiku/SDL_bopengl.cc.o -c /src/src/video/haiku/SDL_bopengl.cc
/src/src/video/haiku/SDL_bopengl.cc: In function 'void* HAIKU_GL_CreateContext(SDL_VideoDevice*, SDL_Window*)':
/src/src/video/haiku/SDL_bopengl.cc:137:21: error: 'BGL_SHARE_CONTEXT' was not declared in this scope
         gl_flags |= BGL_SHARE_CONTEXT;
                     ^
[11/189] Building CXX object CMakeFiles/SDL3-shared.dir/src/video/haiku/SDL_bvideo.cc.o
FAILED: CMakeFiles/SDL3-shared.dir/src/video/haiku/SDL_bvideo.cc.o 
/bin/x86_64-unknown-haiku-g++   -DSDL3_shared_EXPORTS -DSDL_BUILD_MAJOR_VERSION=3 -DSDL_BUILD_MICRO_VERSION=0 -DSDL_BUILD_MINOR_VERSION=0 -DUSING_GENERATED_CONFIG_H -Iinclude-config- -I/src/src -Iinclude -I/src/include -I/src/include/SDL3 -std=c++11 -fPIC -fvisibility=hidden   -D_REENTRANT -Wall -Wno-multichar -Wundef -fno-strict-aliasing -Wshadow -fdiagnostics-color=always -MD -MT CMakeFiles/SDL3-shared.dir/src/video/haiku/SDL_bvideo.cc.o -MF CMakeFiles/SDL3-shared.dir/src/video/haiku/SDL_bvideo.cc.o.d -o CMakeFiles/SDL3-shared.dir/src/video/haiku/SDL_bvideo.cc.o -c /src/src/video/haiku/SDL_bvideo.cc
/src/src/video/haiku/SDL_bvideo.cc: In function 'SDL_Cursor* HAIKU_CreateCursor(SDL_Surface*, int, int)':
/src/src/video/haiku/SDL_bvideo.cc:198:84: error: no matching function for call to 'BCursor::BCursor(BBitmap*&, BPoint)'
         cursor->driverdata = (void *)new BCursor(cursorBitmap, BPoint(hot_x, hot_y));
                                                                                    ^
In file included from /src/src/video/haiku/SDL_BWin.h:40:0,
                 from /src/src/video/haiku/SDL_bvideo.cc:27:
/system/develop/headers/os/app/Cursor.h:52:9: note: candidate: BCursor::BCursor(BMessage*)
         BCursor(BMessage* data);
         ^
/system/develop/headers/os/app/Cursor.h:52:9: note:   candidate expects 1 argument, 2 provided
/system/develop/headers/os/app/Cursor.h:51:9: note: candidate: BCursor::BCursor(BCursorID)
         BCursor(BCursorID id);
         ^
/system/develop/headers/os/app/Cursor.h:51:9: note:   candidate expects 1 argument, 2 provided
/system/develop/headers/os/app/Cursor.h:50:9: note: candidate: BCursor::BCursor(const BCursor&)
         BCursor(const BCursor& other);
         ^
/system/develop/headers/os/app/Cursor.h:50:9: note:   candidate expects 1 argument, 2 provided
/system/develop/headers/os/app/Cursor.h:49:9: note: candidate: BCursor::BCursor(const void*)
         BCursor(const void* cursorData);
         ^
/system/develop/headers/os/app/Cursor.h:49:9: note:   candidate expects 1 argument, 2 provided
/src/src/video/haiku/SDL_bvideo.cc: In function 'int HAIKU_OpenURL(const char*)':
/src/src/video/haiku/SDL_bvideo.cc:299:30: error: 'class BUrl' has no member named 'OpenWithPreferredApplication'
     const status_t rc = burl.OpenWithPreferredApplication(false);
                              ^
ninja: build stopped: subcommand failed.

I'm no Haiku export, so I don't know the correct approach here. (The container has gcc 5.4.0, so it needs -DCMAKE_CXX_FLAGS="-std=c++11")

madebr avatar May 04 '23 03:05 madebr

It's gotta be an old install; modern Haiku uses like...gcc8 or something, so the system frameworks are probably missing some stuff.

icculus avatar May 04 '23 05:05 icculus

https://github.com/libsdl-org/SDL/pull/7677 switches the QEMU emulator with a cross toolchain.

About the build failure for the older toolchain (the docker image was 5 years old), doesn't the Haiku port provide backwards compatibility, or some kind of fallback?

madebr avatar May 04 '23 12:05 madebr

Right now it's not worth it to provide fallbacks for 1.0beta3 installs for stuff they added in 1.0beta4. :)

icculus avatar May 04 '23 12:05 icculus

Is it worth building for 32-bit x86 Linux? I imagine the builders will mostly Just Work with -m32 on the compiler command line (assuming the 32-bit dependencies are installed to make it compile various audio/video/etc backends), but is it something we'd get any value from testing?

Generically, we have other 32-bit platforms in the mix already.

icculus avatar May 04 '23 18:05 icculus

Is it worth building for 32-bit x86 Linux? I imagine the builders will mostly Just Work with -m32 on the compiler command line (assuming the 32-bit dependencies are installed to make it compile various audio/video/etc backends), but is it something we'd get any value from testing?

Looks like we occasionally break it: https://github.com/libsdl-org/SDL/commit/a9c988b2a9f01c0038517ca3a283e3a036eec243 So there is some value.

madebr avatar May 28 '23 15:05 madebr

Hi, is there any chance of building the release artifacts here in the GitHub Actions CI? This would make it easier to reproduce how the windows dll binaries are built.

I am again trying to figure it out why sdl2.dll built from source works and the one shipped in the release (2.30.0 this time) is giving me errors. :/

The VC releases also come with header files and they have been unmatched in the past too.

ericoporto avatar Feb 23 '24 19:02 ericoporto