Add build and test ci

[avinal]

Description

Added build and test CI for metacall/core using GitHub Actions for following platforms

• Ubuntu 20.04, 18.04 GCC and Clang
• Windows 2019 MSVC (failing, need help)
• MacOS 11 Clang (failing, need help)

Fixes #(issue_no)

Type of change

• [x] New feature (non-breaking change which adds functionality)

Checklist:

• [x] I have performed a self-review of my own code.
• [x] I have commented my code, particularly in hard-to-understand areas.
• [x] I have made corresponding changes to the documentation.
• [x] My changes generate no new warnings.
• [x] I have added tests/screenshots (if any) that prove my fix is effective or that my feature works.
• [x] I have tested the tests implicated (if any) by my own code and they pass (make test or ctest -VV -R <test-name>).
• [x] If my change is significant or breaking, I have passed all tests with ./docker-compose.sh build &> output and attached the output.
• [x] I have tested my code with OPTION_BUILD_SANITIZER or ./docker-compose.sh test &> output and OPTION_TEST_MEMORYCHECK.
• [x] I have tested with Helgrind in case my code works with threading.
• [x] I have run make clang-format in order to format my code and my code follows the style guidelines.

If you are unclear about any of the above checks, have a look at our documentation here.

Tests/Logs

https://github.com/avinal/core-1/actions/runs/1921677717

Note for failing ci and suggestions

• MacOS CI is failing because of

  •   fatal error: 'link.h' file not found
      #include <link.h>
                         ^~~~~~~~
    

  • Similiar error: https://github.com/geodynamics/aspect/issues/218
  • Possible solution: https://github.com/geodynamics/aspect/blob/main/CMakeLists.txt#L452-L460
• Windows CI is failing because of
  • https://github.com/avinal/core-1/runs/5390020514?check_suite_focus=true#step:4:1431
  • No suggestions

[avinal]

Hello @viferga, It needs GitHub Actions to be enabled. I also need help with the Windows and macOS CI. Please let me know if you want more features to be added, i.e memory check, coverage etc.

[viferga]

Respect to this PR, I am not sure if we should merge it or not. The problem respect to this is that it is really hard to fully test MetaCall, and we are doing that only in Linux with a dockerized environment right now.

I have added the command ./docker-compose.sh test which does a testing based on sanitizers for detecting memory leak errors and similar.

I don't want to close this PR because it can serve as a base for future testing in other architectures, compilers or operative systems, but I cannot merge it yet because it feels incomplete to me.

Testing the metacall/distributable-{macos,linux,windows} can be also a good option for integration testing if we do not want to suffer installing all the dependencies in the CI for testing.

[avinal]

Hey @viferga, if this seems incomplete then don't merge it, also the overall result that you expect from CI/CD is very different from this PR, but yeah it would be nice to keep this around for future purposes. You can maybe add some appropriate labels to this PR to denote that.

For the integration testing, I think having docker containers with preinstalled dependencies and hosted on GitHub container registry is our best bet. It solves caching problems, reduces CI times by skipping installing dependencies, and you still get all the testing features.

I did something like this for one of my personal project and it reduced the CI duration from 1min 30seconds plus to under 30seconds, https://github.com/avinal/lark/pkgs/container/lark (image)

[avinal]

Meanwhile, I will keep an eye on CI/CD for metacall and will try something better suited to the needs if I get free time.

[viferga]

@avinal Hey mate we are working again on the CI, now Linux is working properly (it has been pretty difficult to implement) and we are working on Windows too. Are you interested in implementing MacOS?

[avinal]

Hi @viferga, I almost forgot that this PR was here 😅. I would love to implement it for macOS but can't promise since now I have a full-time job. Let me try it this weekend. 🤞🏼

[ahmedihabb2]

There are 3 tests that failed on MacOS which is:

• 15- detour-test
• 4 - log-custom-test (Subprocess aborted)
• 35 - metacall-dynlink-path-test

You can view the logs from here https://github.com/ahmedihabb2/core/actions/runs/4126612989/jobs/7128770774

@viferga

[viferga]

@ahmedihabb2 can you create a new PR with your changes? There's things to review in your code.

[ahmedihabb2]

• 15- detour-test
• 4 - log-custom-test (Subprocess aborted)
• 35 - metacall-dynlink-path-test

@ahmedihabb2 can you create a new PR with your changes? There's things to review in your code.

https://github.com/metacall/core/pull/383

[viferga]

@avinal I am closing this issue because as you can see we have advanced a lot with the CI. Also you will notice how complex this task is if you check the current implementation of all scripts and CI for solving it. There's still a lot to improve but at least we have covered a lot of cases and languages of MetaCall for MacOS and Windows. Linux support is complete, including sanitizers. It stills lacks valgrind but the project is not fully prepared to have it right now due to false positives from runtimes and lack of having support to it from the beginning with CI.

[avinal]

Hi @viferga, I understand, and it is a good effort to clear up outdated issues/PR. Thanks