Running an mlx_server on an imported mlx model fails

[jasonnathan]

Running Transformer Lab v0.4.0 on Macbook Pro M1

1. Imported a quantized model converted from huggingface via:

python -m mlx_lm.convert \
--hf-path mistralai/Mistral-Nemo-Instruct-2407 \
-q

2. In the model page, I cannot run the server. I have tried restarting the app. There are no settings to see how the mlx server is run nor can I find any logs.

3. I have tried running the mlx server manually with the same host and port and it works:

% python -m mlx_lm.server \ 
> --host localhost \
> --port 21001 \
> --model ./mlx_model 

UserWarning: mlx_lm.server is not recommended for production as it only implements basic security checks.

2024-08-07 11:08:16,215 - INFO - Starting httpd at localhost on port 21001...

Please see screenshot