pe653
pe653
> > > 两种代码，都没有重复加载kenlm模型，qps在9左右。每秒可以运行9条，不慢。 > > > ``` > > > for s in sents: > > > corrected_sent, detail = correction_model.correct(s) > > > print(corrected_sent, detail ) > >...
> 务启动的时候调用一次， 也可以用 init container，在容器启动之后触发一下服务调用 试了各种方式，最后选择在dockerfile里面加两行指令的方式，在我的kubernetes环境下效果相对最好。镜像layer复用度很好，后面修改代码再次build的速度很快，container里面也不用重新下载model到cache了。就是部署到kubernetes的时候，配置pod时候加一下本地volume ``` /root/.cache /root/.cache/huggingface /root/.cache/torch/ ``` dockerfile里面加 ``` RUN python -c "from sentence_transformers import SentenceTransformer; model = SentenceTransformer('sbert-base-nli')" RUN python -m spacy download zh_core_web_sm ```
Second! Pydantic 2 is really great and lots of langchain's python developers are using FastAPI that bumps to Pydantic 2 recently.
I am handling one the issue myself and will submit a PR request today to solve it. I made testing locally and solved the proxy setting.
> https://github.com/hwchase17/langchain/pull/1453 PR is submitted to solve the issue #1423. Please review it, thanks.
> is there documentation on openAIs side about when/why to do this? openai documentation [error code](https://platform.openai.com/docs/guides/error-codes/python-library-error-types) doesnt explicitly indicate the reason to set openai.proxy for APIConnectionError, while I noticed that...
It seems that I finally figured out the How-to solve the issue. AzureOpenAI sample seems not the answer since not all of attributes are extracted from env setting in "openai/openai-python/init.py"...
@haohao0103 @youngfreeFJS If you use FastAPI server, refer to code as below ```python @asynccontextmanager async def lifespan(app: FastAPI): load_dotenv() # load .env variables, e.g. OPENAI_API_KEY / SERPER_API_KEY openai.proxy = os.getenv("HTTP_PROXY")...
Got same error as below Validating model schema... ``` ⅹ Model schema is invalid: invalid components: schema "Output": extra sibling fields: [title] ⅹ ⅹ {"components":{"schemas":{"Embedding":{"properties":{"embedding":{"items":{"type":"number"},"title":"Embedding","type":"array"},"shape":{"items":{"type":"integer"},"title":"Shape","type":"array"}},"required":["shape","embedding"],"title":"Embedding","type":"object"},"HTTPValidationError":{"properties":{"detail":{"items":{"$ref":"#/components/schemas/ValidationError"},"title":"Detail","type":"array"}},"title":"HTTPValidationError","type":"object"},"Input":{"properties":{"source_image":{"description":"input image file handler","format":"uri","title":"Source Image","type":"string","x-order":0}},"required":["source_image"],"title":"Input","type":"object"},"Output":{"$ref":"#/components/schemas/Embedding","title":"Output"},"PredictionRequest":{"properties":{"created_at":{"format":"date-time","title":"Created At","type":"string"},"id":{"title":"Id","type":"string"},"input":{"$ref":"#/components/schemas/Input"},"output_file_prefix":{"title":"Output...
Got same error when I managed to return numpy embedding file generated by ViT transformer. I am curious how to return a downloadable file on replicate deployment, e.g. COCO format...