Github Actions to build linux & windows binaries?

[m-schuetz]

Would be great to automatically compile windows and linux binaries with github actions. Wasn't able to get it to work with C++20, though.

[potree]

Might actually have to specify "ubuntu-20.04" since "ubuntu-latest" is, in fact, not the latest: https://github.com/actions/virtual-environments

If linux builds won't work, at least windows builds might work using "windows-latest"

[midnight-dev]

20.04 LTS would make the most sense for broad compatibility. Though an explicit apt upgrade/install might be necessary.

Regarding past attempts using GitHub, did you try passing in a flag to the compiler to enable C++20? GCC 10 should compile with the new standard if you pass this to it:

-std=c++20

Alternatively, there are additional GNU extensions on top of the 2020 spec that are enabled with:

-std=gnu++20

For what it's worth, I have yet to try this myself. I've only used VS 2019 locally. In theory, it should be all the same across any platform, but in reality there are small differences in MS C++20 because they added features to their IDE that were not finalized. It's not strictly conformant.

[dddpt]

[EDIT: pull request made, using an improved version of this proposed docker image in pull request 586. In the meantime, it is available in the forked repo.]

Here is a one-stop solution to compile and run PotreeConverter inside a Ubuntu 20.04 docker container. I tested it on Ubuntu 18.04 and 12.04 and Mac OS X, works fine. It doesn't exactly solve the problem of creating binaries/github actions, but it's very handy for all the folks having problems compiling on linux/Mac.

Copy the Dockerfile at the end of this post, it's standalone, and cd to its directory .

Build the docker image (takes quite some time):

$ docker build . -t potreeconverter --network host

Run the docker image it with your las files' folder mounted inside (adapt to your path):

$ docker run -it --rm -v '/absolute/path/to/folder/with/your/las/file/':/PotreeConverter/build/files potreeconverter bash

From inside the container, potree-convert (adapt filenames), and then exit the container:

# ./PotreeConverter files/input_file.las -o files/potree_octree_folder
# exit

chown the output files so you're the owner:

$ sudo chown -R your_username potree_octree_folder

I freezed the versions of some of the utilities/binaries used in this docker. As of 23.11.2020, those are:

• cmake 3.19.0
• tbb git commit https://github.com/wjakob/tbb/commit/141b0e310e1fb552bdca887542c9c1a8544d6503
• PotreeConverter 2.0.2, git commit da93ec26411b82ffbf799daf5ac56608ef988bf8

Feel free to update them to more recent versions, but not guarantee it'll still work.

The Dockerfile:

# build environment
FROM ubuntu:20.04 as ubuntu

# install gcc, openSSL, git
RUN apt-get -y update
RUN apt-get -y install wget
RUN apt-get -y install git
RUN apt-get -y install build-essential
RUN apt-get -y install libssl-dev


# install cmake
RUN wget https://github.com/Kitware/CMake/releases/download/v3.19.0/cmake-3.19.0-Linux-x86_64.tar.gz
RUN tar -xzf cmake*.tar.gz 
RUN mv cmake*/ cmake/

# install tbb
RUN git clone https://github.com/wjakob/tbb.git
WORKDIR /tbb/build
RUN git checkout 141b0e310e1fb552bdca887542c9c1a8544d6503
RUN ../../cmake/bin/cmake ..
RUN make
RUN make install
WORKDIR /


# compile PotreeConverter
RUN git clone https://github.com/potree/PotreeConverter.git
RUN mkdir /PotreeConverter/build
WORKDIR /PotreeConverter/build
RUN git checkout da93ec26411b82ffbf799daf5ac56608ef988bf8
RUN ../../cmake/bin/cmake ..
RUN make

# copy libtbb.so in PotreeConverter build
RUN cp /tbb/build/libtbb.so /PotreeConverter/build/libtbb.so

CMD ["sleep", "infinity"]


# Building:
# ---------------------
# docker build . -t potreeconverter --network host
# -> option --network host needed to access network for apt-get, wget and git
#
# Running docker image and converting a las file to potree format inside docker:
# ---------------------
# docker run -it --rm -v '/absolute/path/to/folder/with/your/las/file/':/PotreeConverter/build/files potreeconverter bash
# ./PotreeConverter files/input_file.las -o files/potree_octree_folder
# -> the potree octree folder will appear in /absolute/path/to/folder/with/your/las/file/potree_octree_folder
# -> outside of docker you will need to set yourself as the owner of the potree folder:
# sudo chown -R your_username potree_octree_folder
#
# Notes:
# ---------------------
# To ensure it works, I freezed the versions of the utilities/binaries used in this docker.
# As of 23.11.2020, those are:
# - cmake 3.19.0
# - tbb git commit 141b0e310e1fb552bdca887542c9c1a8544d6503
# - PotreeConverter 2.0.2, git commit da93ec26411b82ffbf799daf5ac56608ef988bf8
# Feel free to update them to more recent versions.

[saha007]

docker: Error response from daemon: invalid mode: /PotreeConverter/build/files. Any idea why this error is coming up? Thanks.

[dddpt]

When does this happens? on docker build, docker run or when converting?

My guess is it is linked to the mounting of the las file folder in: docker run -it --rm -v '/absolute/path/to/folder/with/your/las/file/':/PotreeConverter/build/files potreeconverter bash

Did you specify a valid path for the las file folder? (in place of '/absolute/path/to/folder/with/your/las/file/') Are you on Windows? you might want to check this SO answer

[midnight-dev]

You absolutely don't have to run with a docker container if you just want to compile binaries on Windows or Linux. To me, it just adds an extra layer of abstraction where problems can occur.

Still, this isn't exactly on the topic of GitHub Actions. If you're still having trouble, search for a related issue or create a new one to help keep things tidy.

[dddpt]

You absolutely don't have to run with a docker container if you just want to compile binaries on Windows or Linux. To me, it just adds an extra layer of abstraction where problems can occur.

You need quite a few packages to compile PotreeConverter (on linux at least), each of which creates a new chance of problem. It seems to me that setting up a list of clunky packages and making sure everything works is exactly the kind of problems docker has been created for and why it's so popular.

Maybe it's easier on Windows. On Linux, getting PotreeConverter to compile took me a week scrolling through issues here, on TBB's github issues or stackoverflow. That's why I created this docker.

So, absent github actions and linux binaries, this docker goes a long way ;-)

[nomandera]

Maybe it's easier on Windows. On Linux, getting PotreeConverter to compile took me a week scrolling through issues here, on TBB's github issues or stackoverflow. That's why I created this docker.

I absolutely have to echo this sentiment. The number of tickets and post related to PotreeConverter compilation issues is daunting, I must have read and tried at least a dozen variants.

At the very least one Linux distro should be supported out of the box and PRs accepted to keep that working.

A far superior option would be to use Github actions but failing that a sane compromise would be a docker container that delivered a portable/static binary.

As it stands the barrier to entry for PotreeConverter is needlessly ... sometimes impossibly... high.

[dddpt]

I made a pull request using an improved version of the above proposed docker image in pull request 586.

In the meantime, it is available in the forked repo.

[nomandera]

@dddpt thanks for putting in the work on this.

Is it possible to use this container as a build environment for PotreeConverter so that the tools could extracted and used directly on a host?

I am in a position where I need to deploy PotreeConverter in an restricted environment that can have some changes made to it but not the free for all that is the current build mess.

(caveat I would need to run this on 18.04 but if the concept is sound I can put the effort into getting this working using your dockerfile as a roadmap)