FastChat

FastChat copied to clipboard

Awesome Jobs guys - Got it work in Docker it's fast on a my 3060 and even faster on my 3080

3

I would like to know how fast, I could locate and setting to see token/s any change this is an option or change info the logger settings ? Awesome job...

CTGS-Innovations

Hi there! I have verified flan-t5 training goes smoothly. Could you review the training file? I am rebasing inference file and will update later.  A sample answer by the...

DachengLi1

Can I further fine-tune on the Vicuna-13B base?

1

theslugger

I use following command to train with my data, model_name_or_path is vicuna-7b-1.1 not llama 7B, Is there any problem with training like this? how to load save step checkpoint-200? Do...

candowu

AMD Support

8

Does this work on AMD cards? What are the GPU requirements for inference?

GrahamboJangles

fastchat.model.apply_delta error

2

Loading checkpoint shards: 100%|███████████████████████████████████████████████████████████████████| 41/41 [00:28

anbo724

any evaluation between 7B and 13B ?

1

i launch the model worker 7B and 13B, it works much worse in many question, how about your's?

NovasWang

out of gpu memory using 4xA100 40G

4

Hi, I used the training script in the readme, and didn't change the data and parameters, but my gpu memory still run out. Have you test it on 4xA100 40Gb?...

puppet101

OpenAI-ish API with batch generation

9

This is a draft PR that serves more informational purpose for anyone wanting to interact with the model in an OpenAI-API-ish fashion. I will not implement additional features upon request...

nielstron

Deepspeed support and config file?

1

Hi: Can this model be trained with Deepspeed support? If yes, could anyone provide a workable Deepspeed config file? Thanks. BTW, I have tried to use a simple config setting...

yzxyzh