Error starting model on HuggingFace inference endpoint

[artificial-paul]

Endpoint encountered an error.
You can try restarting it using the "pause" button above. Check [ logs](https://ui.endpoints.huggingface.co/artificial-paul/endpoints/evo-1-131k-base-artificial/logs) for more details.
Server message:Endpoint failed to start. tory 2024-02-28 05:58:09,130 | INFO | No custom pipeline found at /repository/handler.py 2024-02-28 05:58:09,130 | INFO | Using device GPU Loading /repository requires to execute some code in that repo, you can inspect the content of the repository at https://hf.co//repository. You can dismiss this prompt by passing `trust_remote_code=True`. Traceback (most recent call last): File "/opt/conda/lib/python3.9/site-packages/starlette/routing.py", line 705, in lifespan async with self.lifespan_context(app) as maybe_state: File "/opt/conda/lib/python3.9/site-packages/starlette/routing.py", line 584, in __aenter__ await self._router.startup() File "/opt/conda/lib/python3.9/site-packages/starlette/routing.py", line 682, in startup await handler() File "/app/webservice_starlette.py", line 57, in some_startup_task inference_handler = get_inference_handler_either_custom_or_default_handler(HF_MODEL_DIR, task=HF_TASK) File "/app/huggingface_inference_toolkit/handler.py", line 45, in get_inference_handler_either_custom_or_default_handler return HuggingFaceHandler(model_dir=model_dir, task=task) File "/app/huggingface_inference_toolkit/handler.py", line 17, in __init__ self.pipeline = get_pipeline(model_dir=model_dir, task=task) File "/app/huggingface_inference_toolkit/utils.py", line 261, in get_pipeline hf_******** = pipeline(task=task, model=model_dir, device=device, **kwargs) File "/opt/conda/lib/python3.9/site-packages/transformers/pipelines/__init__.py", line 705, in pipeline config = AutoConfig.from_pretrained(model, _from_pipeline=task, **hub_kwargs, **model_kwargs) File "/opt/conda/lib/python3.9/site-packages/transformers/models/auto/configuration_auto.py", line 986, in from_pretrained trust_remote_code = resolve_trust_remote_code( File "/opt/conda/lib/python3.9/site-packages/transformers/dynamic_module_utils.py", line 538, in resolve_trust_remote_code answer = input( EOFError: EOF when reading a line Application startup failed. Exiting. Do you accept? [y/N]

[Zymrael]

Direct inference on HF is not supported. We recommend running it locally via the HF implementation (StripedHyenaForCausalLM) or using the Together API, which will be live tomorrow

[artificial-paul]

Thanks! Will the HF implementation work on OSX or would I still need the full CUDA+linux dependencies?

[Zymrael]

I think you will need full CUDA+linux dependencies