LLinkedlist

我也是，我之前是美国服务器的，用了一个月了，都没啥事，昨天都封了。

Do not open meaningless issues to bother the maintainer's mailbox.😅

Here are smoe changes I forget to push in the openai api server part. It is related with the `chun.json`. I will commit thoese changes as soon as possible.

> Here are smoe changes I forget to push in the openai api server part. It is related with the `chun.json`. I will commit thoese changes as soon as possible....

@merrymercy Hi, can you review this PR?

This PR has beeen delayed about one month, I am not sure weather the maintainer has seen it yet.

Also I found the API uses a global model to provide the service, it could make my life easier to have a balancer.

Each time you call the `generation` function, it will creat a new model (`load from your disk and init`) , I think it would be the cause of it.

claude的API的system messsage加到函数调用里面了。不在messages里面了，做一个adapter就行了。

You can set the `device_map` when load the `embedding model` with `transformer`.