fairgbm icon indicating copy to clipboard operation
fairgbm copied to clipboard

Correct the compatibility description

Open eustomaqua opened this issue 10 months ago • 5 comments

As people noticed (https://github.com/feedzai/fairgbm/issues/45), fairgbm is not supported on macOS. Besides, I tested fairgbm on Cent OS and RHEL8 systems, and it didn't work as well. So I don't think the statement---and I quote, "Note Compatibility is only maintained with Linux OS."---is correct, because the only Linux where I know it works is Ubuntu, although so many other Linux systems out there and I didn't test them all. Here I attached the errors for your information.

Error info

Running fairgbm on RHEL8 (Linux)

>>> import fairgbm
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "~/miniconda3/envs/ensem/lib/python3.8/site-packages/fairgbm/__init__.py", line 8, in <module>
    from .basic import Booster, Dataset, register_logger
  File "~/miniconda3/envs/ensem/lib/python3.8/site-packages/fairgbm/basic.py", line 95, in <module>
    _LIB = _load_lib()
  File "~/miniconda3/envs/ensem/lib/python3.8/site-packages/fairgbm/basic.py", line 86, in _load_lib
    lib = ctypes.cdll.LoadLibrary(lib_path[0])
  File "~/miniconda3/envs/ensem/lib/python3.8/ctypes/__init__.py", line 451, in LoadLibrary
    return self._dlltype(name)
  File "~/miniconda3/envs/ensem/lib/python3.8/ctypes/__init__.py", line 373, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: /lib64/libm.so.6: version `GLIBC_2.29' not found (required by ~/miniconda3/envs/ensem/lib/python3.8/site-packages/fairgbm/lib_lightgbm.so)
>>>

Running fairgbm on CentOS 7 (Linux)

>>> import fairgbm
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "~/Software/anaconda3/envs/ensem/lib/python3.6/site-packages/fairgbm/__init__.py", line 8, in <module>
    from .basic import Booster, Dataset, register_logger
  File "~/Software/anaconda3/envs/ensem/lib/python3.6/site-packages/fairgbm/basic.py", line 95, in <module>
    _LIB = _load_lib()
  File "~/Software/anaconda3/envs/ensem/lib/python3.6/site-packages/fairgbm/basic.py", line 86, in _load_lib
    lib = ctypes.cdll.LoadLibrary(lib_path[0])
  File "~/Software/anaconda3/envs/ensem/lib/python3.6/ctypes/__init__.py", line 426, in LoadLibrary
    return self._dlltype(name)
  File "~/Software/anaconda3/envs/ensem/lib/python3.6/ctypes/__init__.py", line 348, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: /lib64/libm.so.6: version `GLIBC_2.27' not found (required by ~/Software/anaconda3/envs/ensem/lib/python3.6/site-packages/fairgbm/lib_lightgbm.so)
>>> 

Extra

Running fairgbm on MacOS (Unix)

>>> import lightgbm
>>> import fairgbm
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "~/Software/miniconda3/envs/ensem/lib/python3.8/site-packages/fairgbm/__init__.py", line 8, in <module>
    from .basic import Booster, Dataset, register_logger
  File "~/Software/miniconda3/envs/ensem/lib/python3.8/site-packages/fairgbm/basic.py", line 95, in <module>
    _LIB = _load_lib()
  File "~/Software/miniconda3/envs/ensem/lib/python3.8/site-packages/fairgbm/basic.py", line 86, in _load_lib
    lib = ctypes.cdll.LoadLibrary(lib_path[0])
  File "~/Software/miniconda3/envs/ensem/lib/python3.8/ctypes/__init__.py", line 451, in LoadLibrary
    return self._dlltype(name)
  File "~/Software/miniconda3/envs/ensem/lib/python3.8/ctypes/__init__.py", line 373, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: dlopen(~/Software/miniconda3/envs/ensem/lib/python3.8/site-packages/fairgbm/lib_lightgbm.so, 0x0006): tried: '~/Software/miniconda3/envs/ensem/lib/python3.8/site-packages/fairgbm/lib_lightgbm.so' (not a mach-o file), '/System/Volumes/Preboot/Cryptexes/OS/~/Software/miniconda3/envs/ensem/lib/python3.8/site-packages/fairgbm/lib_lightgbm.so' (no such file), '~/Software/miniconda3/envs/ensem/lib/python3.8/site-packages/fairgbm/lib_lightgbm.so' (not a mach-o file)

eustomaqua avatar Apr 16 '24 19:04 eustomaqua

Sorry, I suddenly realise that it might work on Cent OS and RHEL8, wait a moment and I will update later

------------UPDATE-------------

Seems like it might be able to be solved somehow, but currently I don't know how to do that without the root access. Here is what I did according to stackoverflow:

$ wget -4c https://ftp.gnu.org/gnu/glibc/glibc-2.29.tar.gz
$ tar -zxvf glibc-2.29.tar.gz  # change to glibc-2.27.tar.gz on CentOS
$ cd glibc-2.29
$ mkdir build_dir
$ cd build_dir
$ ../configure --prefix=~/Software/glibc
$ make
$ make install

The installation works on RHEL8 (but didn't work on CentOS because

configure: error: 
*** These critical programs are missing or too old: compiler

Even though glibc is installed on RHEL8, it still reports the same error when running fairgbm, as described above. I think running fairgbm might work if I do it this way

sudo ../configure --prefix=/opt/glibc
sudo make
sudo make install

But unfortunately I don't have the root access. Therefore the only way for me would be modifying the source code of fairgbm. Then I got stuck with modifying "lib_lightgbm.so", I don't know how to do that...

eustomaqua avatar Apr 16 '24 19:04 eustomaqua

glibc is an OS dependent library. I recommend avoiding a manual install as it will most likely break some underlying OS features.

Looking at the base centos:7 image it looks like it ships with glibc-2.17 which will definitely not work:

[root@40cae3eede5e /]# cat /etc/redhat-release
CentOS Linux release 7.9.2009 (AltArch)
[root@40cae3eede5e /]# ldd --version
ldd (GNU libc) 2.17
Copyright (C) 2012 Free Software Foundation, Inc.

@eustomaqua can you please provide more context on how you are trying to build this image or install FairGBM? E.g., by sharing a Dockerfile?

Thanks!

joaoleveiga avatar Apr 20 '24 13:04 joaoleveiga

@joaoleveiga Sorry for the late response, I was trying to use docker (centos:7 and centos:8) and reproduce the error but failed to pull these images.

The bugs I reported at the beginning were happening on the Linux servers (one is Cent OS 7, and the other is RHEL8), not a docker image. I mentioned docker because this is the only way I can successfully use fairgbm, by using a docker image of a Ubuntu system, which is also the reason I think the sentence "works for linux systems" in the document is incorrect.

As for the docker image where I can successfully use fairgbm, I shared it already here and there. This is a purely Ubuntu:latest image, with no extra software or package installed. After creating an image using the Dockerfile, I enter the container to manually install miniconda3 and fairgbm. And that was how I made it work for me. I didn't use the official docker image because the file is too large.

Let me know please if you need any other information. Thanks again for your help!

eustomaqua avatar Apr 25 '24 12:04 eustomaqua

Thanks for the PR!

As far as I can tell, the docker image available here corresponding to this docker file works fine as an alternative to use fairgbm without Ubuntu.

AndreFCruz avatar Sep 10 '24 10:09 AndreFCruz

My intention is mainly to point out that the description is not precise, and secondly to give a Dockerfile that provides minimal dependency, which is the operating system---because in Ubuntu, people can simply use "pip install fairgbm" and no need to use a much larger docker image. Besides, I gave the initial Dockerfile here https://github.com/feedzai/fairgbm/issues/45#issuecomment-1729702512 for the available one

eustomaqua avatar Sep 12 '24 17:09 eustomaqua