Produce external_deps archive with naclsdk ported to python3

[illwieckz]

Produce external_deps archive with naclsdk ported to python3. I only tested with linux.

[DolceTriade]

Can we fix the CI and update the CI to work with python3 to verify this change?

[illwieckz]

We just need to upload the to-be-next archive zip and restart the CI task.

[illwieckz]

So what I did is that I extracted current deps archives, applied the patch, and repackaged. It builds on all CI systems.

[slipher]

After some additional patching the produced binaries seem to work on my machine. Nice job!

[illwieckz]

I imported the patch to have all packages using tar.xz format from #713.

I also implemented the deps-<system>-<arch>-<compiler>_<version>.<ext> format. Note that I added a deps- prefix because it makes easy to manage files this way (for example one can do rm deps-*.tar.xz).

I uploaded ready-to-use packages. I haven't built them myself, what I did is that I used version 6 packages, renamed the folders, applied the python3 patch for naclsdk, and packafed as tar.xz.

If people can try to build the game by hand that would be appreciated, as it looks like @slipher caught some problems the CI didn't.

[illwieckz]

Tried to build NaCl on Mac and didn't get past the configuration stage. From the CMake sub-invocation for NaCl it tried to download deps from https://dl.unvanquished.net/deps/deps--i686-default_7.tar.xz.

I fail to understand how that can happen. 😲️

[illwieckz]

Tried to build NaCl on Mac and didn't get past the configuration stage. From the CMake sub-invocation for NaCl it tried to download deps from https://dl.unvanquished.net/deps/deps--i686-default_7.tar.xz.

I fail to understand how that can happen. astonished

I also got this on Linux, after multiple successful attempts:

Downloading dependencies from 'https://dl.unvanquished.net/deps/deps--i686-default_7.tar.xz'
-- [download 0% complete]
CMake Error at daemon/CMakeLists.txt:458 (message):
  Error downloading
  'https://dl.unvanquished.net/deps/deps--i686-default_7.tar.xz':

That doesn't make sense.

Edit: it works again if I clean-up the build dir. I got the error after a failed build that failed because of a cmake syntax error.

[necessarily-equal]

Minimal testing and looking suggests that it work (Linux / NixOS / python2 only). LGTM

[slipher]

I get the bogus deps URL on Mac when starting from an empty build dir, configuring with BUILD_GAME_NACL=1 and immediately attempting to build the nacl-vms target.

[illwieckz]

I get the bogus deps URL on Mac when starting from an empty build dir, configuring with BUILD_GAME_NACL=1 and immediately attempting to build the nacl-vms target.

When this happen, NACL is set.

I did:

if (WIN32)
    set(DEPS_SYSTEM windows)
elseif (APPLE)
    set(DEPS_SYSTEM macos)
elseif (LINUX)
    set(DEPS_SYSTEM linux)
elseif (NACL)
    set(DEPS_SYSTEM nacl)
endif()

And I got:

  Error downloading
  'https://dl.unvanquished.net/deps/nacl-i686-default_7.tar.xz':

[slipher]

Looks like the way it works now is that the parent cmake instance sets -DDEPS_DIR=xxx and the child instance carefully avoids overwriting this variable in the nacl case.

(Which is not quite right btw; it means that building nacl won't work in a cross-compile build. If you want you could take this opportunity to determine the host system architecture in the nacl case and download the deps for that :stuck_out_tongue:)

[illwieckz]

(Which is not quite right btw; it means that building nacl won't work in a cross-compile build. If you want you could take this opportunity to determine the host system architecture in the nacl case and download the deps for that :stuck_out_tongue:)

I don't really understand that. Anyway I'm surprised the code worked before… How did it worked before?

[slipher]

As I said it's the -DDEPS_DIR=${DEPS_DIR} flag to the child cmake invocation in DaemonGame.cmake. It stopped working because you started to always write the value of DEPS_DIR, instead of leaving it untouched when NACL is on.

[illwieckz]

As I said it's the -DDEPS_DIR=${DEPS_DIR} flag to the child cmake invocation in DaemonGame.cmake. It stopped working because you started to always write the value of DEPS_DIR, instead of leaving it untouched when NACL is on.

Ah right, there was no NACL mention but no if for it so since DEPS_DIR was unset the download code was ignored.

So I assume the added if (NACL) test is the correct way to do it.

[illwieckz]

@slipher look at my latest commit, it makes separate <system>-<arch>-pnacl packages.

The 7a version is temporary (will be 7 if it works).

[illwieckz]

I give up for now for splitting the archives. It looks like it uncovers bugs like looking for system lua when building nexe and things like that.

So I removed the commit that split the archives and such effort would be done later (way later), I don't have time to investigate archive splitting more right now.

I'm considering this ready if it builds.

[illwieckz]

I guess you'll need to squash the commits (other than the .deb one) to make all commits buildable?

I don't like squashing but yes, we better want to have commits buildable, so I squashed.

[slipher]

The nacl patch could have gone into its own commit if you put the Python cmake changes into the squashed commit, since I made it work with Python 2.

LGTM

[illwieckz]

Right, I did that as I prefer that.

I made sure to move the DEPS_VERSION incrementing to the commit changing the archive name and format.

[slipher]

I mean the stuff that looks for Python 2/3 instead of Python 2 needs to be in (or after) the big commit.

[illwieckz]

@slipher, here is what I did:

Except the commit removing the support for the old nacl deb which is just cleaning-up things,

• first commit introduce the code to produce python2/3 compatible deps archive;
• second commit use the new deps archive (with python2/3 compatibility) and other changes (name, format…);
• third commit enable the ability to use python3.

I think it's now ready for merging! 🙂️

[slipher]

The commit for enabling Python 3 says that I am the author which is not right.

Otherwise LGTM

[illwieckz]

🎉️