ipex-llm icon indicating copy to clipboard operation
ipex-llm copied to clipboard

Nano: Setting LD_PRELOAD ERROR during initialization under dev environment

Open Oscilloscope98 opened this issue 1 year ago • 1 comments

Under develop environment (export PYTHONPATH=...), errors will occur when setting LD_PRELOAD during source bigdl-nano-init.

image

Oscilloscope98 avatar Sep 06 '22 02:09 Oscilloscope98

Nano's init script will try to find libtcmalloc.so in two location: ${NANO_DIR}/libs/libtcmalloc.so and ${LIB_DIR}/libtcmalloc.so.

  • ${NANO_DIR} is the directory where you install nano, in your case, it's /home/yuwen/BigDL/python/nano/src/bigdl/nano
  • ${LIB_DIR} is the directory which stores current conda environment's lib, in your case, it's /home/yuwen/miniconda3/envs/temp/lib

When you use export PYTHONPATH=..., the libtcmalloc.so won't be downloaded to /home/yuwen/BigDL/python/nano/src/bigdl/nano/libs/libtcmalloc.so, you can download it and place it there manually.

Or you can use another method to develop:

unset PYTHONPATH
cd /home/yuwen/BigDL/python/nano
python setup.py develop

python setup.py develop will install bigdl-nano as a package from source code, the libtcmalloc.so will be downloaded to right place during the installation, so that nano's init script can find it. It can realize almost the same effect as export PYTHONPATH=...

MeouSker77 avatar Sep 15 '22 09:09 MeouSker77