Eleuther Harness on NVIDIA/AMD doesn't work with locally saved models (with a local path)

[deep1401]

Virtual environment found, using it for evaluation...
Running command: $ /home/transformerlab/.transformerlab/workspace/plugins/common-eleuther-ai-lm-eval-harness/venv/bin/python -m lm_eval --model hf --model_args pretrained=/home/transformerlab/.transformerlab/workspace/models/SmolLM2-135M_dummy-multi,trust_remote_code=True --tasks mmlu_abstract_algebra --device cuda:0 --trust_remote_code --log_samples --limit 0.05 --output_path /home/transformerlab/.transformerlab/workspace/experiments/alpha/evals/SplendidCheetah/395
--Beginning to run evaluations (please wait)...
2025-08-01:14:24:27,787 INFO     [__main__.py:279] Verbosity set to INFO
2025-08-01:14:24:32,267 WARNING  [__main__.py:312]  --limit SHOULD ONLY BE USED FOR TESTING.REAL METRICS SHOULD NOT BE COMPUTED USING LIMIT.
2025-08-01:14:24:32,267 INFO     [__main__.py:364] Passed `--trust_remote_code`, setting environment variable `HF_DATASETS_TRUST_REMOTE_CODE=true`
2025-08-01:14:24:32,267 INFO     [__main__.py:376] Selected Tasks: ['mmlu_abstract_algebra']
2025-08-01:14:24:32,268 INFO     [evaluator.py:164] Setting random seed to 0 | Setting numpy seed to 1234 | Setting torch manual seed to 1234 | Setting fewshot manual seed to 1234
2025-08-01:14:24:32,268 INFO     [evaluator.py:201] Initializing hf model, with arguments: {'pretrained': '/home/transformerlab/.transformerlab/workspace/models/SmolLM2-135M_dummy-multi', 'trust_remote_code': True}
2025-08-01:14:24:32,385 INFO     [huggingface.py:132] Using device 'cuda:0'
2025-08-01:14:24:32,466 INFO     [huggingface.py:369] Model parallel was set to False, max memory was not set, and device map was set to {'': 'cuda:0'}
Traceback (most recent call last):
File "/home/transformerlab/.transformerlab/envs/transformerlab/lib/python3.11/importlib/metadata/__init__.py", line 563, in from_name
return next(cls.discover(name=name))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
StopIteration

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "<frozen runpy>", line 198, in _run_module_as_main
File "<frozen runpy>", line 88, in _run_code
File "/home/transformerlab/.transformerlab/workspace/plugins/common-eleuther-ai-lm-eval-harness/venv/lib/python3.11/site-packages/lm_eval/__main__.py", line 461, in <module>
cli_evaluate()
File "/home/transformerlab/.transformerlab/workspace/plugins/common-eleuther-ai-lm-eval-harness/venv/lib/python3.11/site-packages/lm_eval/__main__.py", line 382, in cli_evaluate
results = evaluator.simple_evaluate(
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/transformerlab/.transformerlab/workspace/plugins/common-eleuther-ai-lm-eval-harness/venv/lib/python3.11/site-packages/lm_eval/utils.py", line 401, in _wrapper
return fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^
File "/home/transformerlab/.transformerlab/worksp