opencompass icon indicating copy to clipboard operation
opencompass copied to clipboard

[Bug] Unable to connect to Hugging Face's MultiPLE-Evaluator

Open qingting-zhxy opened this issue 3 months ago • 0 comments

Prerequisite

Type

I'm evaluating with the officially supported tasks/models/datasets.

Environment

{'CUDA available': True, 'CUDA_HOME': '/usr/local/cuda-12.4', 'GCC': 'gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0', 'GPU 0,1,2,3,4,5,6,7': 'NVIDIA H100 80GB HBM3', 'MMEngine': '0.10.7', 'MUSA available': False, 'NVCC': 'Cuda compilation tools, release 12.4, V12.4.131', 'OpenCV': '4.11.0', 'PyTorch': '2.5.0', 'PyTorch compiling details': 'PyTorch built with:\n' ' - GCC 11.4\n' ' - C++ Version: 201703\n' ' - Intel(R) MKL-DNN v3.5.3 (Git Hash ' '66f0cb9eb66affd2da3bf5f8d897376f04aae6af)\n' ' - OpenMP 201511 (a.k.a. OpenMP 4.5)\n' ' - NNPACK is enabled\n' ' - CPU capability usage: AVX512\n' ' - CUDA Runtime 12.4\n' ' - NVCC architecture flags: ' '-gencode;arch=compute_90,code=sm_90\n' ' - CuDNN 90.1\n' ' - Build settings: BUILD_TYPE=Release, ' 'CUDA_VERSION=12.4, CUDNN_VERSION=9.1.0, ' 'CXX_COMPILER=/usr/bin/c++, CXX_FLAGS= ' '-D_GLIBCXX_USE_CXX11_ABI=1 ' '-fvisibility-inlines-hidden -DUSE_PTHREADPOOL ' '-DNDEBUG -DUSE_KINETO -DLIBKINETO_NOROCTRACER ' '-DLIBKINETO_NOXPUPTI=ON -DUSE_FBGEMM ' '-DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK ' '-DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC ' '-Wall -Wextra -Werror=return-type ' '-Werror=non-virtual-dtor ' '-Werror=range-loop-construct ' '-Werror=bool-operation -Wnarrowing ' '-Wno-missing-field-initializers ' '-Wno-type-limits -Wno-array-bounds ' '-Wno-unknown-pragmas -Wno-unused-parameter ' '-Wno-strict-overflow -Wno-strict-aliasing ' '-Wno-stringop-overflow -Wsuggest-override ' '-Wno-psabi -Wno-error=old-style-cast ' '-Wno-missing-braces -fdiagnostics-color=always ' '-faligned-new -Wno-unused-but-set-variable ' '-Wno-maybe-uninitialized -fno-math-errno ' '-fno-trapping-math -Werror=format ' '-Wno-stringop-overflow, PERF_WITH_AVX=1, ' 'PERF_WITH_AVX2=1, TORCH_VERSION=2.5.0, ' 'USE_CUDA=1, USE_CUDNN=ON, USE_CUSPARSELT=OFF, ' 'USE_EIGEN_FOR_BLAS=ON, USE_EXCEPTION_PTR=1, ' 'USE_GFLAGS=OFF, USE_GLOG=OFF, USE_GLOO=ON, ' 'USE_MKL=OFF, USE_MKLDNN=ON, USE_MPI=OFF, ' 'USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, ' 'USE_ROCM=OFF, USE_ROCM_KERNEL_ASSERT=OFF, \n', 'Python': '3.10.18 (main, Jun 5 2025, 13:14:17) [GCC 11.2.0]', 'lmdeploy': "not installed:No module named 'lmdeploy'", 'numpy_random_seed': 2147483648, 'opencompass': '0.4.2+', 'sys.platform': 'linux', 'transformers': '4.51.3'}

Reproduces the problem - code/configuration sample

from gradio_client import Client

ip_address='https://opencompass-multiple-evaluator.hf.space' client = Client(ip_address)

Reproduces the problem - command or script

None

Reproduces the problem - error message

Loaded as API: https://opencompass-multiple-evaluator.hf.space/ ✔ Traceback (most recent call last): File "/home/workspace/test.py", line 4, in client = Client(ip_address) File "/root/miniconda/envs/torch/lib/python3.10/site-packages/gradio_client/client.py", line 153, in init self.config = self._get_config() File "/root/miniconda/envs/torch/lib/python3.10/site-packages/gradio_client/client.py", line 890, in _get_config return r.json() File "/root/miniconda/envs/torch/lib/python3.10/site-packages/httpx/_models.py", line 766, in json return jsonlib.loads(self.content, **kwargs) File "/root/miniconda/envs/torch/lib/python3.10/json/init.py", line 346, in loads return _default_decoder.decode(s) File "/root/miniconda/envs/torch/lib/python3.10/json/decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/root/miniconda/envs/torch/lib/python3.10/json/decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Other information

The MultiPLE-Evaluator Space on Hugging Face is stuck at "Preparing Space" and cannot be accessed

qingting-zhxy avatar Aug 07 '25 02:08 qingting-zhxy