ModuleNotFoundError: No module named 'pllm_python' when serve_8B.py executed

[dwq370]

step1: launch container

mkdir -p ~/nano-test
docker run --gpus all --net=host --privileged -v /dev/shm:/dev/shm --name nanoflow -v ~/nano-test:/code -it nvcr.io/nvidia/nvhpc:23.11-devel-cuda_multi-ubuntu22.04

step2: Install dependencies

git clone https://github.com/efeslab/Nanoflow.git
cd Nanoflow
chmod +x ./installAnaconda.sh
./installAnaconda.sh
# restart the terminal

./setup.sh

step3: execute serve_8b.py

(base) root@qxzg-4l4:/code/Nanoflow# ./serve.sh
Please login to Hugging Face to cache your token.

    _|    _|  _|    _|    _|_|_|    _|_|_|  _|_|_|  _|      _|    _|_|_|      _|_|_|_|    _|_|      _|_|_|  _|_|_|_|
    _|    _|  _|    _|  _|        _|          _|    _|_|    _|  _|            _|        _|    _|  _|        _|
    _|_|_|_|  _|    _|  _|  _|_|  _|  _|_|    _|    _|  _|  _|  _|  _|_|      _|_|_|    _|_|_|_|  _|        _|_|_|
    _|    _|  _|    _|  _|    _|  _|    _|    _|    _|    _|_|  _|    _|      _|        _|    _|  _|        _|
    _|    _|    _|_|      _|_|_|    _|_|_|  _|_|_|  _|      _|    _|_|_|      _|        _|    _|    _|_|_|  _|_|_|_|

    To log in, `huggingface_hub` requires a token generated from https://huggingface.co/settings/tokens .
Enter your token (input will not be visible): 
Add token as git credential? (Y/n) y
Traceback (most recent call last):
  File "/root/anaconda3/bin/huggingface-cli", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/root/anaconda3/lib/python3.11/site-packages/huggingface_hub/commands/huggingface_cli.py", line 57, in main
    service.run()
  File "/root/anaconda3/lib/python3.11/site-packages/huggingface_hub/commands/user.py", line 153, in run
    login(
  File "/root/anaconda3/lib/python3.11/site-packages/huggingface_hub/utils/_deprecation.py", line 101, in inner_f
    return f(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^
  File "/root/anaconda3/lib/python3.11/site-packages/huggingface_hub/utils/_deprecation.py", line 31, in inner_f
    return f(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^
  File "/root/anaconda3/lib/python3.11/site-packages/huggingface_hub/_login.py", line 130, in login
    interpreter_login(new_session=new_session)
  File "/root/anaconda3/lib/python3.11/site-packages/huggingface_hub/utils/_deprecation.py", line 101, in inner_f
    return f(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^
  File "/root/anaconda3/lib/python3.11/site-packages/huggingface_hub/utils/_deprecation.py", line 31, in inner_f
    return f(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^
  File "/root/anaconda3/lib/python3.11/site-packages/huggingface_hub/_login.py", line 290, in interpreter_login
    _login(token=token, add_to_git_credential=add_to_git_credential)
  File "/root/anaconda3/lib/python3.11/site-packages/huggingface_hub/_login.py", line 404, in _login
    token_info = whoami(token)
                 ^^^^^^^^^^^^^
  File "/root/anaconda3/lib/python3.11/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn
    return fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^
  File "/root/anaconda3/lib/python3.11/site-packages/huggingface_hub/hf_api.py", line 1626, in whoami
    headers=self._build_hf_headers(
            ^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/anaconda3/lib/python3.11/site-packages/huggingface_hub/hf_api.py", line 9090, in _build_hf_headers
    return build_hf_headers(
           ^^^^^^^^^^^^^^^^^
  File "/root/anaconda3/lib/python3.11/site-packages/huggingface_hub/utils/_deprecation.py", line 101, in inner_f
    return f(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^
  File "/root/anaconda3/lib/python3.11/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn
    return fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^
  File "/root/anaconda3/lib/python3.11/site-packages/huggingface_hub/utils/_headers.py", line 126, in build_hf_headers
    token_to_send = get_token_to_send(token)
                    ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/anaconda3/lib/python3.11/site-packages/huggingface_hub/utils/_headers.py", line 159, in get_token_to_send
    raise LocalTokenNotFoundError(
huggingface_hub.errors.LocalTokenNotFoundError: Token is required (`token=True`), but no token found. You need to provide a token or be logged in to Hugging Face with `huggingface-cli login` or `huggingface_hub.login`. See https://huggingface.co/settings/tokens.
Prompt [default: The University of Washington is located]: The University of Washington is located
Decode length [default: 100]: 100
Output file [default: trace.csv]: trace.csv
Select model:
1) llama2-70B
2) llama3-70B
3) llama3.1-70B
4) llama3-8B
5) llama3.1-8B
6) Qwen2-72B
Enter the number corresponding to your model choice: 4
Traceback (most recent call last):
  File "/code/Nanoflow/pipeline/utils/serve_8B.py", line 6, in <module>
    import pllm_python
ModuleNotFoundError: No module named 'pllm_python'
cat: trace.req_words: No such file or directory

[Wazrrr]

We have changed our codebase from c++ to python, which would be more clear to debug. Moreover, we have deleted pllm_python module in the python codebase since it's useful in c++ codebase only.