Direct linking from C

[drewmiller]

Summary

Seeking support for linking LightGBM directly using C compilers.

Motivation

LightGBM may form part of a pipeline of other tools written in a variety of languages targeting a variety of platforms. Direct access via the C programming language provides a lowest common denominator across many platform/language combinations in addition to those already supported.

Description

It naively seems this feature can be implemented with a few macros and little overhead testing given the existing C API support.

References

#4608

[jameslamb]

@drewmiller thanks again for bring this to our attention and for your initial contribution in #4608.

I'd like to take you up on your offer from https://github.com/microsoft/LightGBM/pull/4608#issuecomment-921306558. Could you provide a minimal C program that calls a LightGBM API?

If you can help with that, I can create a continuous integration job in our existing setup to test that this feature remains supported.

[drewmiller]

@jameslamb Happy to help.

[drewmiller]

Alright, here's a first run at the simple C example. My intuition was to carry over the logic, test data, etc. from tests/c_api_test/test_.py. However, there's a bit of numpy and filepath magic going on in those examples which might distract from the purpose of this file, so I opted for the minimal case with arbitrary data embedded in the source. I feel this is keeping more in the traditions of a prospective C programmer. I propose that we would add coverage for the basic C API calls made in tests/c_api_test/test_.py in a C file over time. For now, though, this hopefully serves as a self-contained example and test file. It's light on comments for the moment, but I'm happy to flesh it out further if the team agrees with this initial approach. Style comments are welcome.

To run the attached, run tar xzv test.tar.gz in tests/c_api_test/. Then make && ./c_api_test. No output and 0 exit code means a pass. Otherwise, a failure. It's a standalone file for the moment. Although I haven't had the opportunity to grok this project's build and CI system yet, I'm happy to help plug this in to the build system via PR once the code is settled.

Please note that compiling with -pedantic and -Werror flags has shown that the use of stdbool and snprintf in the existing c_api.h and c_api.c isn't compatible with ANSI C. I'll propose a set of changes to that code separately. For now, you'll note that the Makefile has those options commented out and relies on clang for compilation.

Valgrind flags some "memory leaks" coming from GOMP_parallel, but I believe that is a known issue and/or suppressed in other tests.

[Longcuirong]

Alright, here's a first run at the simple C example. My intuition was to carry over the logic, test data, etc. from tests/c_api_test/test_.py. However, there's a bit of numpy and filepath magic going on in those examples which might distract from the purpose of this file, so I opted for the minimal case with arbitrary data embedded in the source. I feel this is keeping more in the traditions of a prospective C programmer. I propose that we would add coverage for the basic C API calls made in tests/c_api_test/test_.py in a C file over time. For now, though, this hopefully serves as a self-contained example and test file. It's light on comments for the moment, but I'm happy to flesh it out further if the team agrees with this initial approach. Style comments are welcome.

To run the attached, run tar xzv test.tar.gz in tests/c_api_test/. Then make && ./c_api_test. No output and 0 exit code means a pass. Otherwise, a failure. It's a standalone file for the moment. Although I haven't had the opportunity to grok this project's build and CI system yet, I'm happy to help plug this in to the build system via PR once the code is settled.

Please note that compiling with -pedantic and -Werror flags has shown that the use of stdbool and snprintf in the existing c_api.h and c_api.c isn't compatible with ANSI C. I'll propose a set of changes to that code separately. For now, you'll note that the Makefile has those options commented out and relies on clang for compilation.

Valgrind flags some "memory leaks" coming from GOMP_parallel, but I believe that is a known issue and/or suppressed in other tests.

When I run this command，make && ./c_api_test, the file “_lightgbm.lib” can't be found