Cristian Gutiérrez comments

Results 10 comments of


                                            Cristian Gutiérrez

python3 -m fastchat.serve.controller

Google Colab redirects `localhost` to the VM backend, using `127.0.0.1` solves the issue: ```bash !python3 -m fastchat.serve.controller --host 127.0.0.1 --port 8000 ```

ERROR: [Errno 99] error while attempting to bind on address ('::1', 21001, 0, 0): cannot assign requested address

Google Colab redirects `localhost` to the VM backend, using `127.0.0.1` solves the issue: ```bash !python3 -m fastchat.serve.controller --host 127.0.0.1 --port 8000 ```

Colab notebook for demo

Hi @merrymercy ! I have a notebook in which I am able to run the FastChat API in Google Colab free tier. As examples I've included the code snippets for...

UserWarning: User provided device_type of 'cuda', but CUDA is not available. Disabling warnings.warn('User provided device_type of \'cuda\', but CUDA is not available. Disabling')

This keeps happening even though I am passing the 'cpu' parameter: `device = torch.device('cpu')` Is there a way to avoid this error popping off in stderr?

[ERROR]: RuntimeError: "addmm_impl_cpu_" not implemented for 'Half'

Move the model to CUDA, you are probably using `float 16` which is only implemented for GPUs.

Optional seq_len argument to torch.nn.utils.rnn.pad_sequence

was this never added?

Support for Llava/multimodal

Will be huge for sure, thanks for the work!

[Usage] Need to install protobuf in order to run CLI Inference

`pip install protobuf` and you will be able to run it fine.

[Usage] Need to install protobuf in order to run CLI Inference

Agree. The dependency comes from the `transformers` library, not LLaVA. https://github.com/huggingface/transformers/issues/24533

Assertion Error "assert isinstance(item, IndexType.args)" in mmengine in Tutorial

@LucaBernecker Hi, have you found out the problem? It must be something related with 'mps' as if I run it with 'cpu' it correctly performs inference.