lobe-chat
lobe-chat copied to clipboard
lobehub
[Request] 自定义模型 传图 判定与规则
🥰 需求描述
根据测试发现现在 自定义非 OpenAI 的模型名 ，会默认支持 传图与插件
这不太合理，是否支持应该由设置者说了算（既然你们想让有的模型不支持 传图与插件 ）
🧐 解决方案
CUSTOM_MODEL 环境变量 通过某种格式，例如 gpt-4-test(1,1) 括号里表示是否开启 传图 和 插件， 1开 0闭
具体如何规范您们开发者考虑，但是强烈建议允许设置这个
📝 补充信息
No response
Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑🤝🧑👫🧑🏿🤝🧑🏻👩🏾🤝👨🏿👬🏿
🥰 Description of requirements
According to the test, it is found that customizing non-OpenAI model names now supports image transfer and plug-ins by default.
This is unreasonable. Whether it is supported or not should be decided by the setter (since you want some models not to support image transfer and plug-ins)
🧐 Solution
CUSTOM_MODEL environment variable via some format, e.g. gpt-4-test(1,1) The brackets indicate whether to enable image transfer and plug-ins, 1 is on and 0 is off.
It is up to you developers to consider how to regulate it specifically, but it is strongly recommended to allow setting this
📝 Supplementary information
No response
👀 @B3n-AI
Thank you for raising an issue. We will investigate into the matter and get back to you as soon as possible.
Please make sure you have given us as much context as possible.
非常感谢您提交 issue。我们会尽快调查此事，并尽快回复您。 请确保您已经提供了尽可能多的背景信息。
Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑🤝🧑👫🧑🏿🤝🧑🏻👩🏾🤝👨🏿👬🏿
Your suggestion is reasonable, but I don't have much idea on how to configure this syntax, so I just let it go. This is better than not being able to configure it. Can we discuss the appropriate setting syntax?
我个人也觉得在CUSTOM_MODEL里定义比较方便，但是不建议在模型名后面添加，可以在展示名后面添加，这样哪怕有错误也不影响使用，顶多就是展示的名称有点问题，出错了也好排查。
如模型名=展示名(0,0）
当然为了避免有人喜欢在展示名里加括号备注，匹配最后一个括号的内容就行，有设置按设置展示，没设置就默认全开。
如模型名=展示名（0.001元/K tokens）(0,0）
另外DIY的标是不是可以不让展示，反正非DIY的模型名称前都有logo也不难区分，或者统一使用一个表示自定义模型的logo，这样模型列表看起来也会整洁许多，加个DIY部署好给不懂的小白用户使用的时候看到DIY以为是什么野鸡模型哈哈哈
还有既然这两个都自定义的话，那模型的最大tokens是否也可以支持自定义，替换掉DIY的标识。
基于上面的提议：
第一种建议是用四舍五入的方法让他显示几K。
如:
模型名=展示名(0,0,4096）在前端展示名就是模型名 ...4k
模型名=展示名(0,0,16385）在前端展示名就是模型名 ...16k
第二种建议是n*1024或者n*1000的方式，在CUSTOM_MODEL里直接自定义nK，那么该模型最大tokens就是n*1024或者n*1000
模型名=展示名(0,0,4K）在前端展示名就是模型名 ...4k，最大tokens就是4*1024=4096或者4*1000=4000
模型名=展示名(0,0,16K）在前端展示名就是模型名 ...16k，最大tokens就是16*1024=16384或者16*1000=16000
Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑🤝🧑👫🧑🏿🤝🧑🏻👩🏾🤝👨🏿👬🏿
I personally think it is more convenient to define it in CUSTOM_MODEL, but it is not recommended to add it after the model name. You can add it after the display name, so that even if there are errors, it will not affect the use. At most, there is something wrong with the displayed name. , it’s easy to troubleshoot if something goes wrong.
Such as model name = display name (0,0)
Of course, in order to avoid people who like to add parentheses in the display name, just match the content of the last parenthesis. If there is a setting, it will be displayed according to the setting. If there is no setting, it will be fully enabled by default.
For example, model name = display name (0.001 yuan/K tokens) (0,0)
In addition, can the DIY logo not be displayed? Anyway, it is not difficult to distinguish non-DIY model names if they have logos in front of them. Or use a unified logo to represent custom models, so that the model list will look much neater. A DIY is deployed for use by novice users who don’t understand. When I saw DIY, I thought it was some kind of pheasant model hahaha
And since both of these are customized, can the maximum tokens of the model also support customization and replace the DIY logo.
Based on the above proposal:
The first suggestion is to use rounding method to let him display a few K.
like:
Model name=Display name(0,0,4096)The display name on the front end is Model name...4k
Model name=Display name(0,0,16385)The display name on the front end is Model name...16k
The second suggestion is to use n*1024 or n*1000. Customize nK directly in CUSTOM_MODEL. Then the maximum tokens of the model are n*1024 or n*1000
Model name = Display name (0,0,4K) The display name on the front end is Model name...4k, and the maximum tokens are 4*1024=4096 or **4*1000=4000 **
Model name = Display name (0,0,16K) The display name on the front end is Model name...16k, and the maximum tokens are 16*1024=16384 or **16*1000=16000 **
Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑🤝🧑👫🧑🏿🤝🧑🏻👩🏾🤝👨🏿👬🏿
The problem with brackets is that they cannot be expanded, because now there is actually file , which is equivalent to supporting uploading files. If other model capabilities are supported in the future, bracket semantics will be problematic
那如果增加环境变量来设置呢，默认全关，给需要开的模型单独设置。毕竟大多数模型只有基础的聊天能力，这样的话也不用填太多
比如在openai的设置下新增类似这种
CUSTOM_MODELS_VISION
CUSTOM_MODELS_FUNCTIONCALL
CUSTOM_MODELS_FILE
CUSTOM_MODELS_MAXTOKENS
......
反正我觉得环境变量的设置越往后越臃肿是必然的哈哈哈
Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑🤝🧑👫🧑🏿🤝🧑🏻👩🏾🤝👨🏿👬🏿
What if you add environment variables to set them? By default, they are all off and can be set separately for the models that need to be turned on. After all, most models only have basic chat capabilities, so you don’t need to fill in too much.
For example, add something like this under the settings of openai
CUSTOM_MODELS_VISION
CUSTOM_MODELS_FUNCTIONCALL
CUSTOM_MODELS_FILE
CUSTOM_MODELS_MAXTOKENS
...
Anyway, I think it is inevitable that the setting of environment variables will become more cumbersome as time goes by, hahaha
开太多变量不太好，一方面是自定义配置成本高，另外一方面是实现也麻烦，未来维护成本高。
我觉得可能还是用 CUSTOM_MODELS 这个变量。我现在想到的语法是考虑用尖括号+属性描述的方式。
id=displayName<maxToken:vision:fc:file>
尖括号第一个值约定为这个模型的 maxToken 。第二个及以后作为模型的扩展能力，能力与能力之间用冒号 : 作为分隔符，顺序不重要。
举几个例子：
-
chatglm-6b=ChatGLM 6B<4096>：ChatGLM 6B，最大上下文 4k，没有高阶能力； -
spark-v3.5=讯飞星火 v3.5<8192:fc>：讯飞星火 3.5 模型，最大上下文 8k，支持 Function Call； -
gemini-pro-vision=Gemini Pro Vision<16000:vision>：Google 视觉模型，最大上下文 16k，支持图像识别； -
gpt-4-all=ChatGPT Plus<128000:fc:vision:file>，hack 的 ChatGPT Plus 网页版，上下 128k ，支持图像识别、Function Call、文件上传
如果没有带 <> ，那不展示 max token，兜底显示 DIY ，提示缺少 MaxToken。
Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑🤝🧑👫🧑🏿🤝🧑🏻👩🏾🤝👨🏿👬🏿
It is not good to open too many variables. On the one hand, the cost of custom configuration is high, on the other hand, the implementation is also troublesome, and the future maintenance cost is high.
I think it might be better to use the CUSTOM_MODELS variable. The syntax I'm thinking of now is to consider the way described with angle brackets + attributes.
id=displayName<128000:vision:fc:file>
The first value of the angle brackets is agreed to be the maxToken of this model. The second and subsequent ones are the extension capabilities of the model, and the colon : is used as the separator between capabilities.
To give a few examples:
-
chatglm-6b=ChatGLM 6B<4096>: ChatGLM 6B, maximum context 4k, no high-order capabilities; -
spark-v3.5=iFlytek Spark v3.5<8192:fc>: iFlytek Spark 3.5 model, maximum context 8k, supports Function Call; -
gemini-pro-vision=Gemini Pro Vision<16000:vision>: Google vision model, maximum context 16k, supports image recognition; -
gpt-4-all=ChatGPT Plus<128000:vision:fc:file>, hack's ChatGPT Plus web version, 128k upper and lower, supports image recognition, Function Call, file upload
Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑🤝🧑👫🧑🏿🤝🧑🏻👩🏾🤝👨🏿👬🏿
Seconded
Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑🤝🧑👫🧑🏿🤝🧑🏻👩🏾🤝👨🏿👬🏿
What about using configuration files? json or yml
Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑🤝🧑👫🧑🏿🤝🧑🏻👩🏾🤝👨🏿👬🏿
What about using configuration files? json or yml
Configuration files are actually an ideal solution, but I don’t know how to make them compatible in both vercel/docker deployment modes.
用配置文件呢？json或者yml
配置文件其实是比较理想的方案，但我不清楚 怎么在vercel/docker 两种部署模式下都兼容
vercel的话，fork仓库之后直接修改仓库里面的配置文件
docker映射仓库里面的配置文件到主机上的配置文件
不知道这个解决方案怎么样
（目前我自己使用的是docker，verel也不是很清楚）
Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑🤝🧑👫🧑🏿🤝🧑🏻👩🏾🤝👨🏿👬🏿
What about using configuration files? json or yml
Configuration files are actually an ideal solution, but I don’t know how to make them compatible in both vercel/docker deployment modes.
For vercel, directly modify the configuration file in the warehouse after forking the warehouse.
docker maps the configuration files in the warehouse to the configuration files on the host
Don't know what this solution is like
(Currently I am using docker, and I am not very clear about verel)
✅ @bbb3n
This issue is closed, If you have any questions, you can comment and reply.
此问题已经关闭。如果您有任何问题，可以留言并回复。
:tada: This issue has been resolved in version 0.147.8 :tada:
The release is available on:
Your semantic-release bot :package::rocket: