[build issue] cuda12.8 torch 2.7 compatiable issue

Open gaowayne opened this issue 7 months ago • 1 comments

hello expert, could you pleaes help check what is the compatible problem below?

I get 0.4.6 tag and I install from flashinfer latest source, my cuda is 12.8, torch is 2.7

srt = [
    "sglang[runtime_common]",
    "sgl-kernel==0.0.9.post2",
    **"flashinfer_python==0.2.5",**
    **"torch==2.7.0",**
    **"torchvision==0.22.0",**
    "cuda-python",
    "outlines>=0.0.44,<=0.1.11",
    "partial_json_parser",
    "einops",
]

(sglang) root@salab-hpedl380g11-02:/mnt/nvme2n1/wayne/sglang/sglang/python# pip install -e ".[all]"
Obtaining file:///mnt/nvme2n1/wayne/sglang/sglang/python
  Installing build dependencies ... done
  Checking if build backend supports build_editable ... done
  Getting requirements to build editable ... done
  Preparing editable metadata (pyproject.toml) ... done
Requirement already satisfied: aiohttp in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from sglang==0.4.6) (3.11.18)
Requirement already satisfied: requests in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from sglang==0.4.6) (2.32.3)
Requirement already satisfied: tqdm in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from sglang==0.4.6) (4.67.1)
Requirement already satisfied: numpy in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from sglang==0.4.6) (2.1.2)
Requirement already satisfied: IPython in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from sglang==0.4.6) (9.2.0)
Requirement already satisfied: setproctitle in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from sglang==0.4.6) (1.3.6)
Requirement already satisfied: aiohappyeyeballs>=2.3.0 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from aiohttp->sglang==0.4.6) (2.6.1)
Requirement already satisfied: aiosignal>=1.1.2 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from aiohttp->sglang==0.4.6) (1.3.2)
Requirement already satisfied: attrs>=17.3.0 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from aiohttp->sglang==0.4.6) (25.3.0)
Requirement already satisfied: frozenlist>=1.1.1 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from aiohttp->sglang==0.4.6) (1.6.0)
Requirement already satisfied: multidict<7.0,>=4.5 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from aiohttp->sglang==0.4.6) (6.4.3)
Requirement already satisfied: propcache>=0.2.0 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from aiohttp->sglang==0.4.6) (0.3.1)
Requirement already satisfied: yarl<2.0,>=1.17.0 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from aiohttp->sglang==0.4.6) (1.20.0)
Requirement already satisfied: idna>=2.0 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from yarl<2.0,>=1.17.0->aiohttp->sglang==0.4.6) (3.10)
Requirement already satisfied: decorator in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from IPython->sglang==0.4.6) (5.2.1)
Requirement already satisfied: ipython-pygments-lexers in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from IPython->sglang==0.4.6) (1.1.1)
Requirement already satisfied: jedi>=0.16 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from IPython->sglang==0.4.6) (0.19.2)
Requirement already satisfied: matplotlib-inline in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from IPython->sglang==0.4.6) (0.1.7)
Requirement already satisfied: pexpect>4.3 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from IPython->sglang==0.4.6) (4.9.0)
Requirement already satisfied: prompt_toolkit<3.1.0,>=3.0.41 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from IPython->sglang==0.4.6) (3.0.51)
Requirement already satisfied: pygments>=2.4.0 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from IPython->sglang==0.4.6) (2.19.1)
Requirement already satisfied: stack_data in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from IPython->sglang==0.4.6) (0.6.3)
Requirement already satisfied: traitlets>=5.13.0 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from IPython->sglang==0.4.6) (5.14.3)
Requirement already satisfied: wcwidth in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from prompt_toolkit<3.1.0,>=3.0.41->IPython->sglang==0.4.6) (0.2.13)
Requirement already satisfied: parso<0.9.0,>=0.8.4 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from jedi>=0.16->IPython->sglang==0.4.6) (0.8.4)
Requirement already satisfied: ptyprocess>=0.5 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from pexpect>4.3->IPython->sglang==0.4.6) (0.7.0)
Requirement already satisfied: charset-normalizer<4,>=2 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from requests->sglang==0.4.6) (3.4.2)
Requirement already satisfied: urllib3<3,>=1.21.1 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from requests->sglang==0.4.6) (2.4.0)
Requirement already satisfied: certifi>=2017.4.17 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from requests->sglang==0.4.6) (2025.4.26)
Requirement already satisfied: anthropic>=0.20.0 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from sglang==0.4.6) (0.51.0)
Requirement already satisfied: anyio<5,>=3.5.0 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from anthropic>=0.20.0->sglang==0.4.6) (4.9.0)
Requirement already satisfied: distro<2,>=1.7.0 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from anthropic>=0.20.0->sglang==0.4.6) (1.9.0)
Requirement already satisfied: httpx<1,>=0.25.0 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from anthropic>=0.20.0->sglang==0.4.6) (0.28.1)
Requirement already satisfied: jiter<1,>=0.4.0 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from anthropic>=0.20.0->sglang==0.4.6) (0.9.0)
Requirement already satisfied: pydantic<3,>=1.9.0 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from anthropic>=0.20.0->sglang==0.4.6) (2.11.4)
Requirement already satisfied: sniffio in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from anthropic>=0.20.0->sglang==0.4.6) (1.3.1)
Requirement already satisfied: typing-extensions<5,>=4.10 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from anthropic>=0.20.0->sglang==0.4.6) (4.12.2)
Requirement already satisfied: httpcore==1.* in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from httpx<1,>=0.25.0->anthropic>=0.20.0->sglang==0.4.6) (1.0.9)
Requirement already satisfied: h11>=0.16 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from httpcore==1.*->httpx<1,>=0.25.0->anthropic>=0.20.0->sglang==0.4.6) (0.16.0)
Requirement already satisfied: annotated-types>=0.6.0 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from pydantic<3,>=1.9.0->anthropic>=0.20.0->sglang==0.4.6) (0.7.0)
Requirement already satisfied: pydantic-core==2.33.2 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from pydantic<3,>=1.9.0->anthropic>=0.20.0->sglang==0.4.6) (2.33.2)
Requirement already satisfied: typing-inspection>=0.4.0 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from pydantic<3,>=1.9.0->anthropic>=0.20.0->sglang==0.4.6) (0.4.0)
Requirement already satisfied: litellm>=1.0.0 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from sglang==0.4.6) (1.69.0)
Requirement already satisfied: click in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from litellm>=1.0.0->sglang==0.4.6) (8.2.0)
Requirement already satisfied: importlib-metadata>=6.8.0 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from litellm>=1.0.0->sglang==0.4.6) (8.7.0)
Requirement already satisfied: jinja2<4.0.0,>=3.1.2 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from litellm>=1.0.0->sglang==0.4.6) (3.1.4)
Requirement already satisfied: jsonschema<5.0.0,>=4.22.0 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from litellm>=1.0.0->sglang==0.4.6) (4.23.0)
Requirement already satisfied: openai<1.76.0,>=1.68.2 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from litellm>=1.0.0->sglang==0.4.6) (1.75.0)
Requirement already satisfied: python-dotenv>=0.2.0 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from litellm>=1.0.0->sglang==0.4.6) (1.1.0)
Requirement already satisfied: tiktoken>=0.7.0 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from litellm>=1.0.0->sglang==0.4.6) (0.9.0)
Requirement already satisfied: tokenizers in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from litellm>=1.0.0->sglang==0.4.6) (0.21.1)
Requirement already satisfied: MarkupSafe>=2.0 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from jinja2<4.0.0,>=3.1.2->litellm>=1.0.0->sglang==0.4.6) (2.1.5)
Requirement already satisfied: jsonschema-specifications>=2023.03.6 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from jsonschema<5.0.0,>=4.22.0->litellm>=1.0.0->sglang==0.4.6) (2025.4.1)
Requirement already satisfied: referencing>=0.28.4 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from jsonschema<5.0.0,>=4.22.0->litellm>=1.0.0->sglang==0.4.6) (0.36.2)
Requirement already satisfied: rpds-py>=0.7.1 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from jsonschema<5.0.0,>=4.22.0->litellm>=1.0.0->sglang==0.4.6) (0.24.0)
Requirement already satisfied: zipp>=3.20 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from importlib-metadata>=6.8.0->litellm>=1.0.0->sglang==0.4.6) (3.21.0)
Requirement already satisfied: regex>=2022.1.18 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from tiktoken>=0.7.0->litellm>=1.0.0->sglang==0.4.6) (2024.11.6)
Collecting sgl-kernel==0.0.9.post2 (from sglang==0.4.6)
  Downloading sgl_kernel-0.0.9.post2-cp39-abi3-manylinux2014_x86_64.whl.metadata (20 kB)
Requirement already satisfied: flashinfer_python==0.2.5 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from sglang==0.4.6) (0.2.5)
Requirement already satisfied: torch==2.7.0 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from sglang==0.4.6) (2.7.0+cu128)
Requirement already satisfied: torchvision==0.22.0 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from sglang==0.4.6) (0.22.0+cu128)
Requirement already satisfied: cuda-python in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from sglang==0.4.6) (12.9.0)
Requirement already satisfied: outlines<=0.1.11,>=0.0.44 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from sglang==0.4.6) (0.1.11)
Requirement already satisfied: partial_json_parser in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from sglang==0.4.6) (0.2.1.1.post5)
Requirement already satisfied: einops in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from sglang==0.4.6) (0.8.1)
Requirement already satisfied: ninja in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from flashinfer_python==0.2.5->sglang==0.4.6) (1.11.1.4)
Requirement already satisfied: filelock in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from torch==2.7.0->sglang==0.4.6) (3.13.1)
Requirement already satisfied: setuptools in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from torch==2.7.0->sglang==0.4.6) (80.4.0)
Requirement already satisfied: sympy>=1.13.3 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from torch==2.7.0->sglang==0.4.6) (1.13.3)
Requirement already satisfied: networkx in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from torch==2.7.0->sglang==0.4.6) (3.3)
Requirement already satisfied: fsspec in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from torch==2.7.0->sglang==0.4.6) (2024.6.1)
Requirement already satisfied: nvidia-cuda-nvrtc-cu12==12.8.61 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from torch==2.7.0->sglang==0.4.6) (12.8.61)
Requirement already satisfied: nvidia-cuda-runtime-cu12==12.8.57 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from torch==2.7.0->sglang==0.4.6) (12.8.57)
Requirement already satisfied: nvidia-cuda-cupti-cu12==12.8.57 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from torch==2.7.0->sglang==0.4.6) (12.8.57)
Requirement already satisfied: nvidia-cudnn-cu12==9.7.1.26 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from torch==2.7.0->sglang==0.4.6) (9.7.1.26)
Requirement already satisfied: nvidia-cublas-cu12==12.8.3.14 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from torch==2.7.0->sglang==0.4.6) (12.8.3.14)
Requirement already satisfied: nvidia-cufft-cu12==11.3.3.41 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from torch==2.7.0->sglang==0.4.6) (11.3.3.41)
Requirement already satisfied: nvidia-curand-cu12==10.3.9.55 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from torch==2.7.0->sglang==0.4.6) (10.3.9.55)
Requirement already satisfied: nvidia-cusolver-cu12==11.7.2.55 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from torch==2.7.0->sglang==0.4.6) (11.7.2.55)
Requirement already satisfied: nvidia-cusparse-cu12==12.5.7.53 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from torch==2.7.0->sglang==0.4.6) (12.5.7.53)
Requirement already satisfied: nvidia-cusparselt-cu12==0.6.3 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from torch==2.7.0->sglang==0.4.6) (0.6.3)
Requirement already satisfied: nvidia-nccl-cu12==2.26.2 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from torch==2.7.0->sglang==0.4.6) (2.26.2)
Requirement already satisfied: nvidia-nvtx-cu12==12.8.55 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from torch==2.7.0->sglang==0.4.6) (12.8.55)
Requirement already satisfied: nvidia-nvjitlink-cu12==12.8.61 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from torch==2.7.0->sglang==0.4.6) (12.8.61)
Requirement already satisfied: nvidia-cufile-cu12==1.13.0.11 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from torch==2.7.0->sglang==0.4.6) (1.13.0.11)
Requirement already satisfied: triton==3.3.0 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from torch==2.7.0->sglang==0.4.6) (3.3.0)
Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from torchvision==0.22.0->sglang==0.4.6) (11.0.0)
Requirement already satisfied: interegular in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from outlines<=0.1.11,>=0.0.44->sglang==0.4.6) (0.3.3)
Requirement already satisfied: lark in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from outlines<=0.1.11,>=0.0.44->sglang==0.4.6) (1.2.2)
Requirement already satisfied: nest_asyncio in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from outlines<=0.1.11,>=0.0.44->sglang==0.4.6) (1.6.0)
Requirement already satisfied: cloudpickle in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from outlines<=0.1.11,>=0.0.44->sglang==0.4.6) (3.1.1)
Requirement already satisfied: diskcache in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from outlines<=0.1.11,>=0.0.44->sglang==0.4.6) (5.6.3)
Requirement already satisfied: pycountry in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from outlines<=0.1.11,>=0.0.44->sglang==0.4.6) (24.6.1)
Requirement already satisfied: airportsdata in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from outlines<=0.1.11,>=0.0.44->sglang==0.4.6) (20250224)
Requirement already satisfied: outlines_core==0.1.26 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from outlines<=0.1.11,>=0.0.44->sglang==0.4.6) (0.1.26)
Requirement already satisfied: mpmath<1.4,>=1.1.0 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from sympy>=1.13.3->torch==2.7.0->sglang==0.4.6) (1.3.0)
Requirement already satisfied: cuda-bindings~=12.9.0 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from cuda-python->sglang==0.4.6) (12.9.0)
Requirement already satisfied: compressed-tensors in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from sglang==0.4.6) (0.9.4)
Requirement already satisfied: datasets in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from sglang==0.4.6) (3.6.0)
Requirement already satisfied: decord in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from sglang==0.4.6) (0.6.0)
Requirement already satisfied: fastapi in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from sglang==0.4.6) (0.115.12)
Requirement already satisfied: hf_transfer in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from sglang==0.4.6) (0.1.9)
Requirement already satisfied: huggingface_hub in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from sglang==0.4.6) (0.31.1)
Requirement already satisfied: llguidance<0.8.0,>=0.7.11 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from sglang==0.4.6) (0.7.19)
Requirement already satisfied: modelscope in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from sglang==0.4.6) (1.25.0)
Requirement already satisfied: orjson in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from sglang==0.4.6) (3.10.18)
Requirement already satisfied: packaging in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from sglang==0.4.6) (25.0)
Requirement already satisfied: prometheus-client>=0.20.0 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from sglang==0.4.6) (0.21.1)
Requirement already satisfied: psutil in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from sglang==0.4.6) (7.0.0)
Requirement already satisfied: pynvml in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from sglang==0.4.6) (12.0.0)
Requirement already satisfied: python-multipart in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from sglang==0.4.6) (0.0.20)
Requirement already satisfied: pyzmq>=25.1.2 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from sglang==0.4.6) (26.4.0)
Requirement already satisfied: soundfile==0.13.1 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from sglang==0.4.6) (0.13.1)
Requirement already satisfied: torchao>=0.7.0 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from sglang==0.4.6) (0.11.0)
Requirement already satisfied: transformers==4.51.1 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from sglang==0.4.6) (4.51.1)
Requirement already satisfied: uvicorn in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from sglang==0.4.6) (0.34.2)
Requirement already satisfied: uvloop in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from sglang==0.4.6) (0.21.0)
Collecting xgrammar==0.1.17 (from sglang==0.4.6)
  Downloading xgrammar-0.1.17-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.5 kB)
Requirement already satisfied: cffi>=1.0 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from soundfile==0.13.1->sglang==0.4.6) (1.17.1)
Requirement already satisfied: pyyaml>=5.1 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from transformers==4.51.1->sglang==0.4.6) (6.0.2)
Requirement already satisfied: safetensors>=0.4.3 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from transformers==4.51.1->sglang==0.4.6) (0.5.3)
Requirement already satisfied: sentencepiece in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from xgrammar==0.1.17->sglang==0.4.6) (0.2.0)
Collecting nanobind>=2.0.0 (from xgrammar==0.1.17->sglang==0.4.6)
  Downloading nanobind-2.7.0-py3-none-any.whl.metadata (1.8 kB)
Requirement already satisfied: hf-xet<2.0.0,>=1.1.0 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from huggingface_hub->sglang==0.4.6) (1.1.0)
Requirement already satisfied: pycparser in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from cffi>=1.0->soundfile==0.13.1->sglang==0.4.6) (2.22)
Requirement already satisfied: pyarrow>=15.0.0 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from datasets->sglang==0.4.6) (20.0.0)
Requirement already satisfied: dill<0.3.9,>=0.3.0 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from datasets->sglang==0.4.6) (0.3.8)
Requirement already satisfied: pandas in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from datasets->sglang==0.4.6) (2.2.3)
Requirement already satisfied: xxhash in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from datasets->sglang==0.4.6) (3.5.0)
Requirement already satisfied: multiprocess<0.70.17 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from datasets->sglang==0.4.6) (0.70.16)
Requirement already satisfied: starlette<0.47.0,>=0.40.0 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from fastapi->sglang==0.4.6) (0.46.2)
Requirement already satisfied: python-dateutil>=2.8.2 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from pandas->datasets->sglang==0.4.6) (2.9.0.post0)
Requirement already satisfied: pytz>=2020.1 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from pandas->datasets->sglang==0.4.6) (2025.2)
Requirement already satisfied: tzdata>=2022.7 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from pandas->datasets->sglang==0.4.6) (2025.2)
Requirement already satisfied: six>=1.5 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from python-dateutil>=2.8.2->pandas->datasets->sglang==0.4.6) (1.17.0)
Requirement already satisfied: nvidia-ml-py<13.0.0a0,>=12.0.0 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from pynvml->sglang==0.4.6) (12.575.51)
Requirement already satisfied: executing>=1.2.0 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from stack_data->IPython->sglang==0.4.6) (2.2.0)
Requirement already satisfied: asttokens>=2.1.0 in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from stack_data->IPython->sglang==0.4.6) (3.0.0)
Requirement already satisfied: pure-eval in /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages (from stack_data->IPython->sglang==0.4.6) (0.2.3)
Downloading sgl_kernel-0.0.9.post2-cp39-abi3-manylinux2014_x86_64.whl (102.2 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 102.2/102.2 MB 17.4 MB/s eta 0:00:00
Downloading xgrammar-0.1.17-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.3 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.3/4.3 MB 23.2 MB/s eta 0:00:00
Downloading nanobind-2.7.0-py3-none-any.whl (241 kB)
Building wheels for collected packages: sglang
  Building editable for sglang (pyproject.toml) ... done
  Created wheel for sglang: filename=sglang-0.4.6-0.editable-py3-none-any.whl size=4098 sha256=5f2778daa7a3da9ef211a7073cca4e8fdcec44987fda84bafbd14595c56af306
  Stored in directory: /tmp/pip-ephem-wheel-cache-iaiamiuz/wheels/9a/f5/20/c32c9f73958758cce482340602ac68f5e4466529ba7eb0af6c
Successfully built sglang
Installing collected packages: nanobind, sgl-kernel, sglang, xgrammar
  Attempting uninstall: sgl-kernel
    Found existing installation: sgl-kernel 0.1.1
    Uninstalling sgl-kernel-0.1.1:
      Successfully uninstalled sgl-kernel-0.1.1
  Attempting uninstall: sglang
    Found existing installation: sglang 0.4.6.post3
    Uninstalling sglang-0.4.6.post3:
      Successfully uninstalled sglang-0.4.6.post3
  Attempting uninstall: xgrammar
    Found existing installation: xgrammar 0.1.19
    Uninstalling xgrammar-0.1.19:
      Successfully uninstalled xgrammar-0.1.19
Successfully installed nanobind-2.7.0 sgl-kernel-0.0.9.post2 sglang-0.4.6 xgrammar-0.1.17
(sglang) root@salab-hpedl380g11-02:/mnt/nvme2n1/wayne/sglang/sglang/python# python -m sglang.launch_server --model-path mistralai/Mistral-7B-Instruct-v0.3 --port 30000
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/mnt/nvme2n1/wayne/sglang/sglang/python/sglang/launch_server.py", line 6, in <module>
    from sglang.srt.entrypoints.http_server import launch_server
  File "/mnt/nvme2n1/wayne/sglang/sglang/python/sglang/srt/entrypoints/http_server.py", line 45, in <module>
    from sglang.srt.entrypoints.engine import _launch_subprocesses
  File "/mnt/nvme2n1/wayne/sglang/sglang/python/sglang/srt/entrypoints/engine.py", line 42, in <module>
    from sglang.srt.managers.data_parallel_controller import (
  File "/mnt/nvme2n1/wayne/sglang/sglang/python/sglang/srt/managers/data_parallel_controller.py", line 28, in <module>
    from sglang.srt.managers.io_struct import (
  File "/mnt/nvme2n1/wayne/sglang/sglang/python/sglang/srt/managers/io_struct.py", line 31, in <module>
    from sglang.srt.managers.schedule_batch import BaseFinishReason
  File "/mnt/nvme2n1/wayne/sglang/sglang/python/sglang/srt/managers/schedule_batch.py", line 46, in <module>
    from sglang.srt.configs.model_config import ModelConfig
  File "/mnt/nvme2n1/wayne/sglang/sglang/python/sglang/srt/configs/model_config.py", line 26, in <module>
    from sglang.srt.layers.quantization import QUANTIZATION_METHODS
  File "/mnt/nvme2n1/wayne/sglang/sglang/python/sglang/srt/layers/quantization/__init__.py", line 54, in <module>
    from sglang.srt.layers.quantization.awq import AWQConfig
  File "/mnt/nvme2n1/wayne/sglang/sglang/python/sglang/srt/layers/quantization/awq.py", line 18, in <module>
    from sgl_kernel import awq_dequantize
  File "/mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages/sgl_kernel/__init__.py", line 12, in <module>
    from sgl_kernel import common_ops
ImportError: /mnt/nvme2n1/wayne/sglang/lib/python3.12/site-packages/sgl_kernel/common_ops.abi3.so: undefined symbol: _ZN5torch3jit17parseSchemaOrNameERKSsb
(sglang) root@salab-hpedl380g11-02:/mnt/nvme2n1/wayne/sglang/sglang/python#

May 11 '25 12:05 gaowayne

Please try to uninstall flashinfer and use pip install -e "python[all]" to download again.

May 11 '25 18:05 Fridge003

This issue has been automatically closed due to inactivity. Please feel free to reopen it if needed.

Jul 11 '25 00:07 github-actions[bot]